Complete software testing is an important means to evaluate the performance and quality of software, which can be used to check whether the software meets the technical requirements stipulated in the design and project contract, test the ability of software to deal with misoperation, and provide a basis for evaluating the reliability and safety of software.
There are many different classification methods for software testing. According to the design method of test cases, there are white-box testing and black-box testing. According to the test strategy and process, it can be divided into unit test, integration test, confirmation test and system test. From the point of view of whether it is necessary to execute the software under test, it can be divided into static test and dynamic test. All the above methods have been proved to be effective and should be selected according to the purpose of the test.
Communication software testing and general software testing are suitable for the same testing methods, processes and tools, but there are some differences, especially for the testing of mobile phone terminal software. For the testing of mobile phone software, an application system with frequent human-computer interaction and high human requirements, telecom operators need to consider it from two angles: on the one hand, from the developer's point of view, the testing system is required to provide sufficient evidence to prove that the function of the software system is feasible; On the other hand, from the user's point of view, the test system should be required to provide information on the defects and deficiencies of the mobile phone software system.
Second, the difficulties and feasible solutions of mobile phone terminal software testing
It is not easy to evaluate mobile phone software products correctly. On the one hand, the software of mobile phone terminal is directly related to its hardware, and the open interface of hardware may be limited. In addition, the software of each terminal is very different, so it is difficult to directly apply general evaluation tools.
Mobile terminal software is difficult to evaluate, which does not mean that mobile terminal software is unpredictable. Mobile phone software includes two main parts: protocol, man-machine interface and application software/application system. The protocol can only be used after passing FTA (general model certification) (for GSM mobile phones), and its radio resource management, mobility location management and communication management software must meet the standards corresponding to this standard, and the protocol can be tracked and tested by automatic test system. Application software/system is basically suitable for general software testing processes and tools.
How to test the MMI part is the difficulty of testing. Because the interaction between the protocol and MMI can not be completely separated, the problems of MMI software will also affect the protocol part. At present, most companies don't have automated testing software, so they can only simulate users' operation of mobile phones and track problems when they occur. However, some vendors' software is supported by test codes, which can realize automatic testing. The realization method is to develop test code and add it to the original code of mobile phone, but this is not realistic for telecom operators.
For the above problems, one of the feasible solutions at present is software simulation, that is, software testing is carried out through a software simulator, and coverage testing, memory testing, performance testing and consistency testing can be completed on the simulator platform. For MMI, transplanting the mobile phone application to the simulator platform can simulate the display of MMI menu tree, simulate the functions of short messages and telephone calls by simulating network messages, and carry out unit testing and integration testing on some codes. It should be noted that using the simulator to test mobile phone software is not the only test method, but it can often be cross-tested or compared with the same version of the target system.
The second solution is to develop a test system, which provides tracking and analysis of protocol stack and MMI, and simulates the hardware function of users. The system can be controlled by thread calling, including user interface thread, script parsing thread, serial port control thread and device thread. User interface thread provides man-machine interface for testing; The script parsing thread is responsible for parsing the script file and executing the parsed script in response to the control message sent by the user interface thread; The serial port control thread controls the data transmission between the test system and the system under test; The device thread realizes the function of automatically setting device parameters according to the test requirements.
Although the test system can be developed in a modular way, there are still problems of great difficulty in development and great differences in the properties of test objects. A simpler method is to develop based on the existing test platform/system, such as the development of Optitrace suite based on Ageresystem.
Thirdly, the testing of mobile phone terminal software in 3G environment.
In view of the complexity of 3G technology and terminal customization will become the mainstream, the testing of 3G mobile phone software deserves further study. 3G service has been developed abroad for a period of time, and it has accumulated a certain amount in mobile phone testing. At present, the corresponding testing tools have been developed, and the industry is demonstrating the service testing platform.
From the current situation, conformance testing has become a new difficulty in the process of mobile phone software testing in 3G environment. Conformance testing is a testing process to confirm whether the equipment meets the specifications or protocols required by its functions. The conformance testing standard includes three parts: abstract test set (ATS), protocol implementation conformance description (PICS) and protocol implementation additional information (PIXIT). The executable test set is generated on the basis of the above three parts.
Protocol conformance testing and RF conformance testing are the most complicated and important parts. Protocol conformance testing belongs to the category of software testing. In a certain network environment, the protocol implementation under test (IUT) is tested in a black box. By comparing the similarities and differences between the actual output and the expected output of the IUT, the consistency between the IUT and the protocol description is determined, and the success rate of the IUT in the interconnection is determined. In fact, 2G systems also need conformance testing, and 3G systems are more complicated than 2G systems, which makes conformance testing more important.
Protocol conformance test can test whether the signaling protocol between mobile phone and network conforms to TS34. 123 specification. 3gppt34.123 defines about 700 TTCN test cases, which test RLC layer, MAC layer and RRC layer respectively. GCF, the global certification forum, has also formulated the certification process of test case/planning platform and the certification and registration process of end products. According to the priority, the test cases are divided into seven batches, and they are authenticated from high to low.
Judging from the current situation, there are not many vendors that can provide protocol conformance testing equipment, but it is an indispensable part of software testing. When devices are interconnected by multiple manufacturers, conformance testing will enhance people's confidence that devices meet the corresponding protocol standards and increase the probability of interconnection between different implementations of the same standard, which is of practical significance to TD-SCDMA devices with independent intellectual property rights in China.
For 3G mobile phones, apart from the protocol conformance test mentioned above, the testing of MMI and application software/application system will still be the main work in the testing, and with the increasing intelligence of mobile phones, this part of the testing work will bring greater challenges to telecom operators.