M Vijaya Deepti* et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 8, Issue No.

1, 080 - 085

An Overview of Software Testing in Medical System and Medical Device Testing

Department of Computer Science Stratford University, Virginia vijaya_vd2008@yahoo.co.in

M Vijaya Deepti

Department of Electronics and Communication Engineering Andhra University, India muvvala_sai@yahoo.co.in process but to leave the bulk of the testing effort until the project nears completion. Although this is the most common method for developing and delivering products, it is fraught with danger. First, it is difficult to estimate the time and effort it will take to remove all defects once a product is completed, and an inaccurate estimation may lead to shipping delays. Second, the cost of finding problems at the end of a project may be extremely high if fixing them means reworking (or worse yet, redesigning) parts of the application.

M N V S S Kumar

Keywords: Qualification Testing, Verification, Validation, Water-fall Model.

A
INTRODUCTION

I.

Health care professionals rely on software to simulate anatomical and physiological elements of the human body for purposes of training, prototyping, and decision making. Software can also be used to simulate medical processes and protocols to measure cost effectiveness and resource utilization. Whereas much of the software engineering research into simulation software focuses on validation (determining that the simulation accurately models real-world activity), to date there has been little investigation into the testing of simulation software itself, that is, the ability to effectively search for errors in the implementation. Developers and testers must ensure that newly developed products or product enhancements meet functional and performance requirements and that those products are reliable and able to operate consistently under peak loads [1]. The traditional approach for development projects is to perform some unit testing throughout the development

IJ
ISSN: 2230-7818

@ 2011 http://www.ijaest.iserp.org. All rights Reserved.

ES

Abstract--- The development of reliable software is given utmost importance now-a-days especially in the field of medicine where reliability of the software developed is achieved through what is called as testing. Testing is a process of verifying and validating the software to ensure that the specifications and requirements of the end user are met and that it is working according to the system design. Software testing of the medical devices is increasing important to check the efficacy and safety of the devices before they are implemented on the patients in real time. History has seen many disastrous medical system failures which lead to the death of a large number of people. So medical practitioners make it a mandatory to check the medical devices and system to ensure the proper delivery of the tangible product to the customer. In this paper an attempt is made to explain the various steps involved in testing of the medical devices and what elements have to be tested mainly. It also explains about the learning experiences about software testing and its management plans which include the important phases of software verification and validation.

The developers must outline four plans by the end of the user requirements (UR) review to define how the development will be managed in the Software Requirements (SR) phase. These are the: Software Project Management Plan Software Verification and Validation Plan Software Quality Assurance Plan Software project management plan Every project must produce a software project management plan. The plan should declare the objectives of the project. These may take the form of statements of deliverables and delivery dates, or of statements of required functionality, performance and quality. The objectives should be prioritized. SQA staff should examine the statement of objectives and check that all plans are consistent with them. The plan should define the life cycle approach to be used in the project, such as waterfall, incremental delivery or evolutionary. The choice of life cycle approach is crucial to the success of the project. SQA should check that the life cycle approach is appropriate and has been correctly tailored to the project. A process model, based upon the selected life cycle approach should be described in the plan. The process model defines all the major activities. For each activity it should define: entry criteria; inputs; tasks; outputs; exit criteria.

T
II. MANAGEMENT PLANS

Page 80

M Vijaya Deepti* et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 8, Issue No. 1, 080 - 085

Software verification and validation plan Verification It is the process of determining whether the products of a given phase of the software development cycle fulfill the requirements established during the previous phase or the process of evaluating software at the end of the software development process to ensure compliance with the software requirements. Validation V&V plays a large role throughout the Waterfall Model of phased software development. It is essential to the development of quality medical software [2]. During the Requirements phase, a plan is produced for establishing verification and validation of the medical software. A method for configuration control is established. All documents are reviewed for compatibility with the V&V plan. Verification will be the overriding activity at all stages of the software development. Validation will be addressed in the final stages. During the Design phase, the Architecture is compared, or traced, to the Systems Requirements. A Hazard Analysis is produced that identifies the known risk areas of the software. This analysis is critical to the development of quality medical software. By identifying the areas of software whose failure could cause the potential for serious injury or death, the test plan can be modified to ensure correctness [3]. These identified critical areas should always receive additional and mention in the V&V plan During the Implementation phase, the code is verified as it is produced. In order to ensure completeness, the code is traced to the design, linking design specification to coded module. All verification defects are resolved. By the end of the UR phase, the software verification and validation plan needs to include a plan for verifying the outputs of the SR phase and an acceptance test plan. The intention of software Verification is to make sure that the work products are working properly and consistently. Verification techniques used in the SR phase include reviewing, formal proof and tracing. Plans for reviews should include intermediate and formal reviews. Intermediate reviews are used to verify products, and later to clear up problems before formal review. The primary formal review is the Software Requirements Review. SQA would also evaluate the Verification processes by finding the bugs and defects that were missed by SQC (Software Quality Control) during Verification. SQA should examine the software verification and validation plan and check that the review procedures are well-defined, the right people are involved, and that enough reviews are held [4]. Documents that have not been reviewed are just as likely to contain problems as code that has not been tested. SQA staff should examine the software

IJ
ISSN: 2230-7818

@ 2011 http://www.ijaest.iserp.org. All rights Reserved.

ES
B. Qualification testing

verification and validation plan and check that the verification approach is sufficiently rigorous to assure the correctness of the software requirements document. Formal methods for specification and verification of software are often necessary when safety and security issues are involved. The software verification and validation plan needs to define how user requirements will be traced to software requirements. SQA staffs often have to check the traceability matrix later in the project, and it is important that traceability procedures are easy to use [5]. Verification answers the queries like are we building the product correctly and Validation answers the question are we building the correct product. Validation explains that the end product gratifies the intended purpose when placed in the right environment while Verification refers to the actual developing of the product according to specification The term Acceptance Test could also apply to Validation, in most cases the Acceptance test is carried out by a different group of people from the SQC team that performed Verification, as the product was being built.

A. Unit and Integration testing Since the accuracy of the dose calculation algorithms is the main safety issue in a treatment planning system, the unit testing was started from them. It was not enough to test the user interface only since the correctness of the calculation phases had to be checked also in intermediate levels. The software qualification engineer rebuilt the dose calculation algorithms from the specifications without consulting how they were implemented in the actual code to be tested. The rebuilt dose algorithms were compared with the actual code by using various input data sets. This approach introduced the need for so called stubs, i.e., code that had to be written in order to simulate the data that the function under test needed, and drivers, code that called the functions under test [6]. The output data from both the reference code and code under test was then compared automatically. The same approach was used at two higher integration levels by combining the smallest test units into larger entities.

Software requirement specification (SRS) was used as the starting point for the development of qualification test specifications. Applications functionality was grouped to testable entities and test cases were created to test every feature specified in the SRS to ensure the conformance of the system. More tests were created based on the results of hazard analysis to focus on safety issues and the users manual was used to create

T
III. METHODS

Page 81

M Vijaya Deepti* et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 8, Issue No. 1, 080 - 085

IV. ELEMENTS OF A MEDICAL DEVICE TO BE TESTED

In testing a medical device, there are five major elements to cover. The first is the display of the unit. The graphical display must be tested to ensure all pixels are working, and the output on the display is correct for the functioning of the device. Vision tools are used to perform this type of testing. The second area is battery testing [7]. Most devices require battery power even if only for data backup purposes. Some modular instrumentation including Digital Multimeters and Switching are used to perform battery tests. The third area is power. The unit must be tested to ensure the user does not receive a shock from the power supply. The fourth area is Electromagnetic Emissions/Immunity testing. These tests ensure that the medical device does not radiate EMI or malfunctions from susceptibility to EMI. RF tools are used to perform pre-compliance testing. Finally, many medical devices use electronics that must be tested for functionality. A set of modular instruments that can test the electronics in medical device are given as follows. The digital thermometer is an example of a medical device requiring testing. The tests include the following: 1. Displays LCD displays must be verified for pixel on/off, brightness, and contrast.

IJ
ISSN: 2230-7818

@ 2011 http://www.ijaest.iserp.org. All rights Reserved.

ES
Test Objectives

test specifications about the usability of the whole system. Writing these test specifications took about two man months. The test input data of functional tests were created by using the equivalence class partitioning: The input data was divided into classes that were expected to behave in the similar way and one value of each class was used as an input for the test. Also the boundary and other special values were used as test data. Testing was done with both valid and invalid values. Furthermore, real clinical data was used to test the usability. The testing of dose calculation algorithm accuracy was continued in the qualification phase: by comparing the output with published experimental and simulation results, and physical measurements. This way the suitability of both the chosen calculation model and its parameters was verified. Customer acceptance procedures, quality assurance procedures, the users manual and installation guides were tested, either by reading them through or using them as a user would do. Attention was paid also to the usability aspects and communication between the user and the application. All the faults found were entered into a discrepancy report database. Tests were rerun every time there was a reason to believe that the changes in the new version of the application could cause a change in the function under test. Before the final release all the tests were rerun once more.

V. TESTING OF MEDICAL SOFTWARE

In the previous section we outlined a rigorous software development methodology suitable for medical software [8]. In the present section we amplify on requirements for the testing of medical software. First we introduce some useful distinctions: - Testing is nominally the process of executing the software and comparing its behavior to expected results and predefined criteria. -Testing may be performed by the developers and/or the client. - Acceptance is the process of determining whether or not a software product satisfies contract requirements. Clients make acceptance decisions. Certification is the process of stipulating that a software product has required properties that may not be fully testable. Certification may be performed by an organization other than the developer or client. Testing often provides the only basis for acceptance and certification. In this section we review the objectives of testing, discuss test methods, and comment on the current stateof-the-art in software testing with examples drawn from our experience in testing medical software.

In a narrow sense the objective of software testing is the discovery and elimination of "bugs." There are many examples of serious losses attributed to software bugs in supposedly "safe" systems. A software bug is suspected in a recent series of deaths in therapeutic radiology. Bitter experience with programs of even modest size quickly teaches that the total elimination of

2. Battery current/voltage draws must be tested as well as the life cycle of the battery under maximum usage conditions. 3. Temperature accuracy the accuracy of the temperature readout must be calibrated and measured. 4. EMC electromagnetic emissions must be tested to ensure the unit is not radiating EMC signals beyond the allocated spectrum. 5. Internal Self Tests must be performed to verify proper functioning. 6. Computer interfaces for those devices designed to connect to a computer, the interfacing (serial, parallel, etc) must be tested. Other outputs if the digital thermometer generates analog voltage, analog current, frequency, or alarm signals, these must be tested for accuracy. In addition to testing the basic functionality of the device, other tests may be performed. These include safety, environmental testing such as shock, vibration, thermal, humidity testing, etc, biocompatibility and sterility testing, and life-cycle testing.

Page 82

M Vijaya Deepti* et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 8, Issue No. 1, 080 - 085

such errors is impractical. Even if all of the bugs could be detected and eliminated there would remain other areas of concern. Major test objectives are defined in terms of the specific tasks or functions which the software must perform. Criteria include (1) the ability to perform the task completely and (2) the ability to perform the task efficiently (quickly or with a minimum of storage) as well as (3) the ability to perform the task free of errors. Other test criteria may be economic. For example, the testing of prototypes may establish that the fully developed system will be too costly [9]. This is particularly true of the fast-paced commercial medical software market. Test Methodology We have emphasized the importance of methodology in evaluating "good practice" for medical software. When testing complete builds or conducting complete system testing three approaches may be characterized and they are structural, functional and random. These techniques provide complementary strategies for developing test cases. Functional testing methods typically require the preparation of a matrix of numbered functional and/or performance requirements vs. tests where it is established. Random testing methods proceed from the recognition that in practical software systems, the number of possible program paths grows rapidly beyond the ability of the tester to exercise exhaustively. But a meaningful sample of the system behavior may be obtained with a test generator that varies the input in a random manner. This approach is more helpful when the aim is to probe vulnerability to security breaches, system crashes, or system degradation rather than the accuracy of calculated results. The approach is also useful in validating software reliability models based on statistical arguments

VI. TESTING PACEMAKERS AND DEFIBRILLATORS A defibrillator is a device implanted into the body and under cardiac arrest conditions provides a shock to the heart to reestablish a beat. It also provides a pulse generation to pace the heart. A pacemaker provides primarily the later function. Pacemakers and defibrillators consist of complex digital circuitry that must interface to its exterior environment through analog connections. A telemetry channel provides interfacing for device diagnostic purposes after implantation. Defibrillators must provide a kilovolt level shock to the heart without damaging the internal circuitry of the device. Because the device is implanted it must have a substantial battery life. The test challenge relates to making precision measurements on complex digital circuitry that has been stimulating and interfaced through analog signals. While this is not a traditional mixed-signal test, signal analysis is a key element of testing the device. The second challenge is the wide range of signals to measure. The shock voltage reaches kilovolts and amperes, but must run on nanoamps and sense microvolts. For pacemakers, the primary function is generating a pace pulse to ensure continuous heart beats. This pace pulse consists of a waveform signal, whose rise time, drop, and recharge pulse amplitude yields information about the condition of the digital circuitry. Figure 1 shows a graphical representation of the pulse signal to be measured. Test instrumentation for the pace pulse test consists of a signal source and a digitizer as shown in Figure 2. The signal source provides the stimulus representing the heartbeat. A digitizer reads the response from the device representing the pacing pulse from the device to the heart. The sensing test uses the signal source to generate Haversine waveforms that simulate the electrical activity of a heartbeat [7].

IJ
ISSN: 2230-7818

Test Conduct

From the beginning of computer programming it was recognized that the testing phase, in particular, is as much an art as it is a science. This recognition proceeds from the simple fact that even for small systems the number of possible program paths is far too large for the tester to exhaust. Thus the tester may design a test plan in which the requirements vs. tests matrix is complete, that is, every requirement is tested at least once, but in the test description or test procedure it quickly becomes apparent that only a small fraction of all possible combinations of input values and conditions is exercised. Recognition of this situation has led to the concept of random testing mentioned previously. Refinements in the concept of random testing attempt to attach priorities to test cases on the basis of frequencyof-use, severity of error and other criteria. Such an

@ 2011 http://www.ijaest.iserp.org. All rights Reserved.

ES

approach is crucial for the large software systems under development today. To reduce the chance for bias, system testing is today generally performed by an independent test team. A subset of the system tests is usually chosen as the basis for customer acceptance tests when the acceptance tests are not designed by the customer. Alternatively, the customer may contract with an independent testing organization to design and perform acceptance tests. The independent test team lies outside the line authority of the project's system engineering and software development departments. Moreover, a product assurance office oversees the testing organization and in a rigorous environment, the product assurance office will function in parallel with the project management and report above the project level.

Page 83

M Vijaya Deepti* et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 8, Issue No. 1, 080 - 085

External shock tests the sensing circuitry after an external shock has been applied.
Recharge Amplitude Reference Pulse Width (50% Point) Pulse Amplitude Pace Pulse Drop (As % of Pulse) Recharge Pulse

VII. LESSONS FROM MEDICAL SOFTWARE TESTING EXPERIENCE Testing medical software includes testing for an automated blood chemistry analyzer, a picture archiving and communications system, a medical data network, a telephony system which records messages for radiologists, and large hospital information systems. This convinced that medical software practitioners must follow a rigorous engineering methodology with strong project management to deliver safe and effective systems. Factors like the personnel assigned to the projects, the complexity of the hardware/software interface, and the computer language used are different for each system. While specific problems are unique, certain classes of problems tend to recur. In particular, it was found that: 1. Cost and schedule overruns in earlier parts of the project limited the funds and time available for testing 2. The end product was not exactly what the endusers wanted even though it rigidly met the original specifications and 3. Small technical problems caused large cost and schedule impacts when not controlled. Examples of these problems and solutions are discussed below. Most of them could have been avoided by adopting a rigorous engineering approach to software development.

IJ
Switching Digitizer
Sense Amp

High Voltage Output

PG Lead

A
Defibrillator Scaling Isolation Circuit Signal Source

This simulates the condition in which an external defibrillator has generated a shock to a patient with a pacemaker implanted. To perform this test, relays are used to switch high voltage capacitors onto the leads. With an external voltage applied, the device is tested to confirm its ability to detect heart rhythms. Additional tests include the following: 1-Battery Voltage 2-Lead impedance 3-Safety core 4-Gross Fail Shock 5-Timm Leads 6-Sense Amp Differential Gain 7-Sense Amp Common Mode 8-Temperature Measurement 9-Battery usage 10-Sensitivity Refractory Testing

Figure 2: Instrument Setup for Pacemaker Test

ISSN: 2230-7818

@ 2011 http://www.ijaest.iserp.org. All rights Reserved.

ES
VIII.
[1] [2] [3] [4]

Software testing is very important phase in the software development cycle because it checks whether the particular software design is free of bugs and defects. Although all software developers face common cost and quality concerns, medical software poses special challenges to developers and regulators. So software testing reduces the cost and time consumption by detecting the errors in advance. Software testing is very important in the field of medicine as an error in medical instruments costs a persons life. Medical device testing is becoming increasingly important as it determines the safety and efficacy of devices. So an overview of software testing is brought out here and it is clearly explained for testing medical devices by considering examples like Pacemakers and Defibrillators. REFERENCES
Testing Implantable Medical Devices, J. Max Cortner, Guidant, Evaluation Engineering, June 2004, page 38. Y.-S. Ma, J. Ofutt, and Y. R. Kwon. MuJava: An automated class mutation system. Journal of SoftwareTesting, Verification and Reliability, 15(2):97{133, June 2005. B. Beizer, Black Box Testing, New York, Wiley, 1995. C. Kaner, Software Negligence and Testing Coverage, Software QA Quarterly, Volume 2, No. 2,

T
CONCLUSION

Figure 1: Pace pulse from a pacemaker.

Page 84

M Vijaya Deepti* et al. / (IJAEST) INTERNATIONAL JOURNAL OF ADVANCED ENGINEERING SCIENCES AND TECHNOLOGIES Vol No. 8, Issue No. 1, 080 - 085

[5] [6]

[7]

[8] [9]

Van Dyk J., Barnett R.B., Cygler J.E., Shragge P.C., Commissioning and quality assurance on treatment planning computers, Int. J. Radiat. Oncol. Biol. Phys. 26: 261473,1993 J. Callahan, F. Schneider, and S. Easterbrook. Automated software testing using model-checking, Technical Report NASA-IVV-96-022, NASA Independent Verification and Validation Facility, Fairmont, WV, August 1996. C. Murphy, G. Kaiser, L. Hu, and L. Wu. Properties of machine learning applications for use in metamorphic testing. In Proc. of the 20th International Conference on Software Engineering and Knowledge Engineering (SEKE), pages 867{872, 2008. E. J. Weyuker. On testing non-testable programs. Computer Journal, 25(4):465{470, November 1982. Model-based testing and validation of control software with reactis, Technical report, November 2003

IJ
ISSN: 2230-7818 @ 2011 http://www.ijaest.iserp.org. All rights Reserved. Page 85

ES