Testing Strategy for Healthcare Software

Learn how others are accelerating their testing cycles

Sara:Alright, good morning and welcome to our webinar, Testing Strategy for Healthcare Software. My name is Sara-Lynn Brunner and I will be your host today. Today’s presenter is Disha Thakkar. Deesha is a senior automation architect for Apexon. Deesha had held various IT and management roles in banking and engineering industries with experience in business requirement and documentation, issue logging and tracking. She holds an MBA in Information Technology Project Management from Symbiosis International University, and a Bachelor of Engineering Degree in Computer Science as well as certifications from financial markets and commercial banking. Disha:Thank you, Sara. Good morning to everyone and thank you again for joining us. This is a generation of – or this is the era of technology and digitization. And me, as a technical cementation [00:01:31] we are continually looking for excellent medical care facility in the real time. But along with that, we are also looking for the healthcare information anywhere, any time. So these days, we as the patients are always wanting to connect to our healthcare providers and we also want to share our personal healthcare information with the hospitals, most elaborate clinicians, to everybody immediately on our fingertips. So with this, let’s move onto the agenda for this presentation or for this webinar. So first of all, we will be looking into the healthcare software testing, and the compliance that revolves around this healthcare software testing. We’re looking to the implications of the regulatory bodies and the compliance agencies on which have been part of the healthcare software or the product. Next, we’ll look into the testing strategy and the testing approach for the regulatory, compliance for this healthcare software which is compliant with the regulatory bodies such as FDA, HIPAA, Meaningful Youth, PHI compliance, etc. Next, we’ll also look at some of our accessories based on the Apexon experience and expertise in the testing and the healthcare domain. Lastly, we’ll answer all the questions in the Q&A section. So, way back in the ‘50s, there was this first wave of early IT adoption that happened in the healthcare industry. What idea adoption there was, there was many processes in the healthcare system that were very repetitive as well as they were very highly standardized. So this first wave of IT adoption tried to automate this repetitive processes. Now 20 years later, we faced the second wave of idea adoption in the healthcare industry. The second wave of IT adoption started to integrate all the standalone disparate systems and it started to bring this whole end to end process into one idea. To give you an example, just take an example of hospital Bill in the accounting was talking to HR, and then HR was also talking to the payroll system. What we are looking at today actually is this is the generation of the third wave of IT adoption. The third wave of IT adoption is fully into the digitization of this whole healthcare ecosystem. Many say that we want to digitize this whole ecosystem, we mean that if a patient is entering, or if a patient is enrolling themselves into the hospital, so starting from taking the appointment to the last results in the end, this whole system is being digitized. What it is doing right now is it is bringing into the analytical data. Now, this analytical data is very, very important, already crucial at this point. What we have observed throughout our experience in testing and the healthcare industries, this first wave and the second wave of IT adoption was more or less concentrating on the processes. Whereas the third wave of IT adoption that we are looking at right now is focusing more onto the patient. So because of this digitization of healthcare that is going on currently, we have identified an Apexon that there are these five basic – there will be five main aspects that will be ruling the healthcare, and these five rules will be patient-centric. The first one is the interoperability. It is very, very important that you need to have a seamless interoperability between two disparate systems. Now, consider a situation where a patient is visiting a clinic. Now, this patient’s information is being recorded into original form if now the patient is given that existing IP data format, and then a CCD change report is being produced at the end of the visit. Just think of the complexity between this different technology standard that comes into the picture. So as an application, you need to have a holistic approach towards your healthcare ecosystem, whereas you have a very improved interoperability between the different integration of the third party systems, as well as you are also covering the technology of all the technical standards. In addition to this interoperability, currently the government has also passed the Macro Act. Now, what this macro is doing is it is trying to emphasize – or it is trying to bring three interoperable systems. That is the patient within the system, and the visitor’s account. Now, these two systems really brought into one single framework, and then this one system will be integrated with the payment system. Next is the integration of security and the privacy. Here we are talking about the digitization of healthcare to such great extent that counts regulatory such as FDA, HIPAA, FMLA, ADA, etc. Now, you are talking about a patient whose information is being transmitted over the internet to the cloud, or it is being stored into the smart device, or in some cases it is stored in the medical device. Just think of the security aspects that is covering this information. So you need to have the regulatory as well as the compliance parties that will support this whole THI aspect. Such as the self help, now there is this very driving force towards healthcare, also you have got a long waiting line for the appointment. So this has reverted into the digitization of this whole ecosystem. We are predicting that there will be, in the near future, there will be healthcare consumers who will be looking into a variety of healthcare providers for their healthcare needs. And these needs will be created on their own with the digital ecosystems. One such example, if we have to think about is telecom medicine. So what exactly is telemedicine? So telemedicine is just a digital platform that will connect physicians with the patients remotely And, your patient still provides videos or they can also send the photos of whatever problems or progress that they are facing. And in addition to that, the physicians will also give them the diagnosis. Next, it is very, very important that if the cloud and data analytics that could be surrounding the PHI, that is the Protected Healthcare Information. So now your cloud technology will be very, very instrumental in providing a controlled access to the hospitals as well as to the health network. US Department of Human and UN Health Services, that is the DHHS, they have set out guidelines for certain cloud service providers. These cloud service providers are the famous names such as Amazon web services and Microsoft edge. Now, they have to bind or they have to connect with the HIPAA compliance, and then they have to find with HIPAA because of business agreement so that they can give the secured EPH to the healthcare industry, and then this can be accessed to over the healthcare network. The last one is the proliferation of IoT variables. When we talk about the smart device, now smart devices have got evolved into the variables. These days, or in the past few years, we have seen lots and lots of emerging technologies around the IOT with the help of variables. IOT is Internet of Things. So then the data is generated between disconnected devices, this data becomes very, very important for the healthcare providers to make very accurate and informed decisions about the patients. Now, using this data, healthcare providers can diagnosis as well as they can give the cure to the patient – they can give cure to the patients immediately. Also, these variables are currently used in monitoring the patients in the clinical trials. This has been – this is becoming very significant in terms of the clinical trials are not able to be with study participants 24/7 to gather all data. They can gather all the data that is related to the clinical trials from the variables, and then this data is put into good use of compliance issues. So this is the area where we are talking about the patient who is looking for all the solutions. You have to put your patient in the center, and you have to devise all of your solutions around data to be successful in providing the solutions that patient is looking for. So it becomes very, very crucial for the healthcare organizations to find the different ways of serving them, as well as they have to be made ready for all of this digitization in the healthcare solutions. But when we are talking of these digital transformations in the healthcare industry, regulatory bodies and compliance agencies come into picture. Now, what does regulatory bodies are doing? They are enabling the patients and the stakeholders to share their poster health information with the trust that this will not get the information being disclosed to the party that is not supposed to be. And then when they are using this digital platform for the healthcare problems, they are blessed with better solutions and better usability. So in the interest of time, in this webinar we will be focusing more onto FDA and HIPAA compliance, but there are other compliances as well, such as Meaningful Use stage one, stage two, and stage three. For that also we are offering our services. So just to start with, what FDA is? The FDA is Food and Drug Administration, and it is a U.S. federal agency that is responsible for regulating the drugs, biological products, and the medical devices. For the bulk of this webinar, we will be concentrating more onto the FDA verification and validation. Now, these guidelines, when we talk about the software aspect of the medical device, it is applicable to the four companies of the healthcare software. The first is a software that becomes a part, component or accessory of a medical device. So take an example. You have a blood glucose monitoring device. And that blood glucose monitoring device is communicating with your mobile application. So that mobile application will become the component of your medical device. Next is the medical device in itself. For your blood glucose device monitoring, would also be having some software residing inside it. So, that will also come under the purview of FDA bnv. Third is the software that will be used in the production of medical device which could be a programmable controller or logic system. And the last is the quality management system that will be implemented at the device manufacturer’s unit. Next, if we talk about HIPAA, so what exactly HIPAA is? It is just a set of generally accepted security standards and requirements that are present for protecting the public – or for protecting the health information. Now, HIPAA is having two titles, Title I and Title II. For this webinar, we’ll be concentrating only on Title II because it revolves around the security and the privacy aspects of the software. So for that I have categorized HIPAA security policy into four quotients. First is the user authentication. So, let’s take an example. A user who is authorized to log into the healthcare software, only that user can log in. Next is the user authorization. So if a user has logged into it, but there would be some information; only that information should be visible to the user. So in user authorization you check on that only the particular necessary information is only visible to the user. Third, it’s very important that if the audit trail so all the transactions that will happen between your healthcare software that comes under the HIPAA, they all need to be accessed with the proper set of audit trail. And the last with the speed data transfer. So you need to ensure the data encryption at all the transfer points that will happen in your healthcare software. So these are some of the testing that we can apply on the healthcare software. Functional testing being you have to test a functional capability of the healthcare application, which would include all the healthcare uploads across the enterprise level. Next, if you have to talk about the person who healthcare providers need to provide the support to the patients 24/7 if they are thinking of the digital platform available to the patient. So, you need to ensure that your healthcare software is not down even a single minute in this 24/7. As part of compatibility, you need to check your applications on various mobile platforms which could be IOS, Android, etc. And if you are talking of the mobile web, then check the applications on multiple browsers. In terms of security testing, your most of the healthcare enterprise applications are noted with very critical person’s information that involve around the patient’s health. So, this could include your health records, payment information, account details, and identity details. So, due to the incredible sensitivity of this data, security testing at this point is not a luxury but it has become a necessity for the healthcare applications to be tested. The last one is the external integration. In most of the cases, what will happen is your healthcare software would be definitely talking to the third party application. So if it then was payment integration, then it will be talking to bank or the payment gateway. And the example that I gave about the blood glucose monitoring device, in that case your mobile application or your software is talking to the medical devices. So, you need to ensure the whole integration that stems between this system. So, in this webinar, what we will be doing is we will showcase some of the testing strategy and process for healthcare software of the products that is either FDA compliant, or HIPAA compliant, or both of these compliant. Okay, so we have the traditional concept of testing, that your testing needs to be done either where you have some development features ready with you, or once your development cycle is done. So in this case, what happens is your manual test will start writing test cases, then they will execute the manual testing of test cases and then they will log the detection and then write the results. After all of this manual testing process is done, then they will develop – freeze your development, and then it will be built and deployed. After this whole development cycle plus your testing cycle and your production cycle gets finished, this is the time when you will start automating. You start to identify the test cases that you want to automate. You will develop the test groups and the execution reporting takes place. So as we found out in our polling question, that long testing cycle is nothing is bugging you more when you are testing the healthcare software that is entirely the compliant, it is because of this reason or it is because of this approach that it is taking so much amount of time. Now, let’s look at the better approach in which you can reduce your long testing cycles. So what we propose is let’s have a agile sprint testing concept. Why only to have agile development, but let us also test in an agile way. So what we are proposing is that you can test your verification and validation activities from the very start of the project. You have identified, or you have written down your manual test cases. At the same time, you identify your test cases that need to be automated. So just imagine you are developing, and you are developing the application and in addition to that you are also developing your test scripts. Now, both things are done finally. So when you have to – when you are exactly following to your testing cycle, at the time what you have to do is just to execute this automated test script. In this, also you are testing in the testing phase. But it will be very quicker and it will be very faster. And what you are doing is you are testing – you are sprinkling your testing parts at each and every iteration of your project development life cycle. With this, the most benefit that you will get out of this is it will increase your quality. It will also shorten your long testing cycle which has already been the main problem for the healthcare industry. And it will also reduce the possibility of getting the unpleasant surprises in terms of bugs and issues that you will find out in the end of development life cycle. But how exactly you will implement this whole agile in sprint testing concept when you are talking about testing your healthcare application or healthcare software, that is complying to a very stringent regulatory body. So what will happen is that you first of all, just take an example that you are a medical device manufacturer. Now, you being a medical device manufacturer, testing is a form of all business or a whole part of your organization. So in most of the cases, a medical device manufacturer will take the help of a vendor who is there and to who will be testing – who is there into the test management tool, or who is providing you the tools for your automation. So there are really two tools you will be looking at. First is the test kit automation tool, and the other is your test management tool. Your test management tool, based on the compliance, the compliance says that your test tool should also be validated and verified. Now, you are a medical device manufacturer who is not able to test their own software, but now they are informed that they need to test the third party vendor test management tool. So that is not going to be possible for this device manufacturer. Now, this is when that how and what type of tool you are selecting is also very important. In some cases, what happens is there has been – what happens is your product or your healthcare software, which is depending on the medical device, gets rejected because your test management tool is not FDA compliant or it is not following the guidelines that is set down into the FDA. This test management tool also forms a very important tool when you have to write down or you have to note your test cases into this tool. You will be – as the testing company, there will be the test cases that will be returned in the test management tool, and then there will be defects that will be logged into the test management tool. It is required that you have to check your requirements versus your design applications. And when these defects come out, you need to ensure that these defects follow a report that is compliant-driven. Now, these compliant-driven reports are too much into the physical paperwork. So what happens is this process becomes a very cumbersome and very complex and tedious and error-proof. What we try to do is we try to automate this close to last stage, and using the test automation tool that we have in house developed that geometric automation frameowork we try to automate the results that needs to come out in a specific format that is the FDA performed. Also, in some cases what happens is the test lead who is in charge of your testing, they need to review and be assigning authority, and this will take around approximately on our experience, it takes around one to two hours for the test lead to check the 200 page of the test report that comes up every day to sign and get it reviewed. One other aspect of the compliance that comes into the picture is whenever there is any change in results to software, there needs to be taken a validation analysis. Now, this validation analysis is not only placed on the individual change, but it also says that you need to check the impact of this change on the entire software system. So let’s take an example. There is a change of one line at one portion in one feature of your software. Then you need to change, or you need to validate and verify this change over the whole ecosystem of this testing. So just imagine if you are doing it manually, how much amount of time it will take. Take an example that a 45 steps scenarios are to be tested on seven devices; then you are doing it manually. Then, if you approximately take around ten minutes to do it. But what if we automate the whole process that is from identification of your test cases to your maintenance. Then, this process will – you can do this 5times a week and each day it will only take three to four hours just to get this quantify scenarios active in dependence on the seven devices. So, the impact that it will bring when you bring into the test automation and the quality engineering, it is truly a significant level. So let’s look at some of the success stories which we have had in the – based on our experience, exposure, and expertise in the testing, and the healthcare industry. So the first success story that I am going to discuss about is a client who is into the digital technology innovation in the healthcare. So, they invented thisdigital medicine, which is an adjustable sensor. And this adjustable sensor is talking to the variable patch. This variable patch in turn communicates with the smart device using the VNA or Bluetooth technology together on the healthcare related necessary information. Now, for this client we are responsible for performing the testing of the mobile application that is residing on the smart device. Also, we are also responsible for testing the variable device that is communicating using this VNA technology. Now, this being an ingestible product, this product is FDA approved, and so the software that is revolving around this FDA approved product, it also needs to follow certain rules and guidelines as part of this compliance. So when we started the thing for them, based on the FDA rules and regulations, there was a need that we need to record the evidence in terms of what is the test results that we have got against each test case, and against each tested. So this was totally a manual process. Now, by manual process what I mean is there was lots and lots of physical paperwork all over. Now, this physical paper used to contain the test cases. Then there were test steps under each test case, and then in front of each test case we need to put down digital that whether that this test has passed or failed, along with the screenshot of the analysis. Now, this result used to come out – it used to come around 200 to 250-page report. And each day our team used to take the printout offered. The team lead used to sign each and every page and then scan it and send it over to the client. So, what happened is all the steps then became a very, very tedious task, and started to piling up into the complexity of this project. What we did was we found out that FDA is approving the electronic signature. So, our team prepared the test chart automation scripts that was based on this icon language. And then, at the end of the execution of this test case, one test file was being generated that was used to export this test through Jira. Now, in Jira using the extra plug-in, these test results that were generated were sent to the device manufacturer, to our client, and then it used to get under the FDA guidelines. This activity reduced our daily manual oriented task drastically and this whole solution became paperless, and it was much more efficient. Testing started to concentrate more on testing rather than signing and reviewing this difficult paperwork that was containing the test results. The next project that we did was for a client who is an American multi national medical device pharmaceutical and consumer packaged goods manufacturer. Now, they had this blood glucose monitoring device. So that was a medical device that was talking to a mobile application sitting on this smart device. So we were part of testing this mobile application, which was integrating with this medical device. So as part of our testing strategy, what we had to do was this product, this medical device was FDA as well as it was HIPAA compliant. Now, they have this stringent law about using a specific test management tool as well as a specific test automation framework. As part of the test management tool, we had to use the Html tool because HP was able to comply with the guidelines that were set under the FDA as well as set under the HIPAA. Also, in the in-house QMetry automation framework tool that was based on selenium, that was being used to test automate the whole end to end solution. The problem here was that there are different levels of reviewing and approval. So the first step of reviewing and approval was followed into – was to be performed by the QA team, or by the testing team that was implementing. So they have this test set. This test set were containing the guest test just for those and the test procedures. What we did was we always started to implement the agile and sprint testing strategy from the very first day of the software development. Then, this all reviewing and approval process was automated so that our testers were able to concentrate more onto the testing and these results were taken care from the very first day. The client was not able to – initially, the client was not able to get or to realize the benefit of the test automation that they had previous to Apexon. But because we implemented in the CIDC using Jenkins, this test automation realization came into picture. The third and the last success story that we would like to discuss is about the client who is a leading healthcare provider in America. For them, what we did was this was not a medical device, or it was not FDA approved healthcare software. But what it was doing was it was HIPAA compliant. And because it was HIPAA compliant, we had to follow some of the PHI related compliance. We had to perform area testing that the American Disability Act as part of the PHI compliance. As part of protected health information, we performed entitlement testing. The entitlement testing, what we did was we gave some of the user-based control testing in which a step of the automated this whole testing in which some of the user data was put into, and based on the combination of the pin and the combination of the medical record number, if a user has to be given a proper access to specific information, the user was given that information. So this whole process or this whole testing process of this entitlement of the user specific control was done automatically. The key takeaways that I would like to give for this webinar is that compliance forms a very, very important aspect when you talk about the testing in the healthcare industry. And one of the major challenges that is faced by the healthcare organizations is the tedious, cumbersome, difficult paperwork. With our testing strategy, we try to eliminate this physical paperwork by automating the reporting process in the required format. This results in reduction of the humanly made errors drastically. Also, our testing strategy helps the healthcare software companies to accelerate the testing cycle. And then it allows them to focus more on the core business rather than the testing business. Verification and validation of the software is taken care of using this strategy. With the help of the shift left strategy that we had discussed earlier, our focus on testing becomes very effective and efficient. And then we are quickly able to deliver the testing results in the compliance desired format. Also, we can achieve quality by testing in short amount of time. But to achieve the quality at speed when compliance comes into the picture, then it becomes very difficult and quite complex to achieve. So this is where we are helping the organizations who are looking for testing the healthcare software with the highest quality in the fastest speed. Sara:And thank you, Disha. At this time we would like to get your questions answered. And as a reminder, you can ask at any time the questions in the Q&A window at the center top of your screen. It looks like the first question is FDA is all about verification and validation of the software. How is verification and validation separately done for healthcare software under FDA? Deesha? Disha:Thanks for the question. Now, for the compliance industry, this becomes very, very important question. If you talk about the general testing world, we normally use verification and validation interchangeably. But if we go into the integrated of the this term in the compliance industry, verification is testing and confirming that the output of the software in particular phase of your project development life cycle is as intended and as desired. But how do we know what validation is doing if validation checks whether the final software is what was expected in the requirement phase. So the main difference here is verification looks for the intermediate stage checking of the software, whereas the validation controls the final software product. So these days most of the projects are agile-based projects. So with that, our testing has also become agile. But in the in screen testing strategies that I just showcased, what we do is we keep on testing our software for each and every bit in this plane. So we are verifying the software in that one particular space as well as part of the validation process, we also check the final software that we have deployed on the server is at the expectations by checking the requirement; what is the final software that is there. Sara:Great, thank you Disha. And the second question is what do you cover as part of accessibility testing ADA as part of compliance? Disha:Okay, so by ADA it is American Disability Act. So there is also similar section 508, that talks about the agencies that is needed to provide people with disabilities some equal access to all the electronic information. So as part of accessibility testing that covers the ADA act, we can perform the vital functions as well as the speak selections, as well as the magnification issues and some of the color contrast and the font selection. I hope that I have answered that question. Sara:Okay, great. Thank you. Another question is how do you take care of PHI aspect of healthcare software? Disha:Okay, so this is a very interesting question in terms of compliance because PHI is all about the HIPAA compliance. And PHI is the Protective Health Information. So as part of the HIPAA compliance, you need to ensure that your users’ information is secured and you are not disclosing any of your patients’ personal and private information to the people who need not be looking at it. So let’s take an example of hospital decides that only your medical staff needs to be involved in the care of a patient. So only that person needs to apply to that. So what we do in this case is we have got a few of the ability to use automated test cases for the healthcare software. And as part of that, we take care of the user-controlled actions and the access central aspect of this PHI. Sara:Okay, great. And we have another question for you. If you are testing very early and often in sprints, how do you manage the high number of deviations? Disha:Okay, so this is a very, very good question because generally what happens is in the strategy that we have discussed, people have this thought in their minds that it can relate to a very, very high cost because you have to employ most of the people from the very start of the project to the end of project. In the long run, if you think you have got your test automation ready with you as part of your in sprint testing. So you have already done the automation. In the end, whenever you have to do the testing, you just have to exit your data automation. Also one needs to realize that the bugs that are found at the very latest stage of the project development life cycle are the most costly to fix. So if you find out the bugs that are there that you can find out in the very earliest stage of your project development life cycle, that will save your cost to a very, very high extent, not only cost but it will also save your time and effort of your whole team. So in this way it will be very cost efficient. Sara:Great, thank you Disha. And another question, can we reach required maturity level in testing healthcare software while considering compliance complexities? Disha:Okay, so I guess by maturity level we are talking about like different maturity levels in the continuous delivery model, like the base level, beginner level, intermediate level, or advanced level So we have categorized, or we have the different maturity models when we talk about the testing industry. So if you are a healthcare software or your healthcare company is totally into the manual testing, this is categorized as the base or the entry level of the testing. But if you have some amount, like 20, 30 percent of the automation, then you are considered as the beginner. So if you are considering or talking about that kind of maturity level, then it’s very, very important for you to reach at the advanced level. And the testing strategy that we have proposed here, that testing strategy can help you to reach to the advanced level of the healthcare software by doing the test automation from the very start of the project development life cycle. Sara:Great, thank you Disha. That’s all the questions that we have right now, and thank you everyone for joining us today. You’ll receive an email in the next 24 hours of the slide presentation and a link to the webcast replay. If you have any questions, please contact us at info@apexon.com, or call 1-408-727-1100 to speak with a representative. Thank you so much.