EETimes on 3GSM WCDMA Handset Conformance Testing
At the conclusion of this article, I've clipped some excerpts from 3GPP's Technical Specification Group (TSG #22) meetings in December 2003 in Hawaii, USA, that deal directly with 'testing,' and relate to the references to 3GPP's TS 34.121 made in the article.
>> Testing Times for 3G Players
John Walko
CommsDesign
Mar 15, 2004
commsdesign.com
Jorma Ollila, chairman and chief executive of the world's most successful supplier of cellular phones, invoked the laws of physics last month when deflecting criticism that Nokia and other handset makers are delaying the launch of 3G services in Europe because phones are unavailable or not up to scratch.
Speaking at the giant 3GSM World Congress in Cannes, France, Ollila was responding to an equally big hitter in the mobile industry: Arun Sarin, CEO of the largest operator, Vodafone. The handsets now out there, Sarin told 3GSM conference goers, are "bulky, they get hot and they don't have good battery life. The experience today is unacceptable to our customers."
Caught right in the middle of this blame game between third-generation cellular equipment makers and service providers is test. "There may be some truth in what Nokia and the other handset makers say, but it seems an easy and cheap excuse to just blame the testing regime," said Adrian Scrase, chief technology officer at the European Telecommunications Standards Institute (ETSI).
The whole regimen for testing and validating handsets to comply with tough, hugely complex and still not completely bedded-down specifications is where the 3G rollout rests right now. Test cases must be written and must go through several loops before they can be certified and validated. They need to meet many protocols and to ensure interoperability among networks as well as handover compliance to existing Global System for Mobile Communications and General Packet Radio Service nets.
Moreover, even a test case that has already proven its merit for a particular handset needs to be looked at again as the different releases for Universal Mobile Telecommunications System (UMTS) networks kick in. Hundreds of test cases covering the RF and protocol functions have been validated for Release 99, and the focus now is on the Third Generation Project Partnership (3GPP) TS 34.121 test cases, Release March 03.
To achieve a known and reliable level of 3G performance, thorough testing is essential. And on one thing all are agreed: This can't be done on an ad hoc basis, but only through defined, industry-standard tests. Certified test cases must be used, just as they are for GSM.
The really big problem is testing for intersystem handover, mainly in the GSM-to-3G direction. Oddly, this vital aspect of the process has been a low priority, both on the network side and for radio access technology.
No wonder Nokia chief Ollila bemoaned the chicken-and-egg situation the handset makers find themselves in. To test handsets, several stable networks must exist. But because of the complexity of 3G handsets, this is taking time, Ollila said. "The waiting period for handsets has been quite reasonable when you consider the laws of physics and how the technology works."
Indeed, at the higher operational frequencies of 3G, and given the addition of audio and video, designers find it tough to stay within the bounds of the laws of physics to compensate for such unwanted phenomena as increased crosstalk, deteriorated signal integrity and so on. No wonder, then, that it has taken longer than expected to get phones to market.
But time is something operators like MMO2, T-Mobile, Orange and others that joined Vodafone's Sarin in criticizing handset quality don't have. They need to start recouping some of the billions they paid over three years ago
to operate third-generation services. Hutchison has started a 3G service in the U.K. and Italy, using handsets that have not been tested to the same rigorous regime that will be mandatory by the end of this year. But the mainstream operators are now talking of "soft" launches in the summer and some kind of commercial service by early next year.
Most test equipment makers and test houses contacted accept that deadlines are being stretched. But they are not willing to take all the blame. "Certainly, there is a great deal more to be done, but people should also recognize what has been achieved," said ETSI's Scrase.
Representatives at equipment companies such as Agilent, Anite, Anritsu, Racal Instruments Wireless Solutions and Spirent Communications--and at test houses like BABT and Radio Frequency Investigation Ltd.--note that several organizations are involved in the test quest. The most important is the terminals group within the international 3GPP, which oversees the setting of specifications for all 3G-related technologies.
For testing, this group works closely with the Global Certification Forum (GCF), a voluntary organization that nevertheless has huge influence, because it is largely controlled by the handset makers and network operators. Between them, the 3GPP-T1 organization and the GCF are responsible for agreeing on the test cases that handsets must conform to before they can be certified for use.
For ETSI's Scrase, the situation is all too familiar. "The same thing happened ahead of the introduction of GSM, and look what a success that became," he said. "And it will probably happen as the cellular industry shifts to the next generation. It is a hugely complicated process, not merely changing the air interface, but catering for video and voice, for color and monochrome displays, and coping with any number of protocols and form factors."
At this point, he said, "getting the test regime right and everyone getting behind [it] . . . is more important than rushing things through." The process is so complex that "it is just unreasonable to expect test suites to be ready at the same time as the phone designs. A proven description of a test just has to be stable before it can be applied to a particular phone design. And there is not anything we can do about this time lag."
The industry, Scrase said, "is spending huge amounts on the process. Our budget alone here at ETSI is euros 1 million [about $1 million] a year, and has been for the past three years, and I expect it will be running at the same rate for the next few years. And that does not take into account all the voluntary input in terms of manpower from the handset manufacturers, network operators and test equipment technologists."
Meanwhile, the number of industry-standard 3GPP test cases being approved and validated is increasing all the time. There are about 800 of these, and last year the GCF started prioritizing them in an attempt to speed up the process. The latest batch was due to get the nod last week at the 3GPP Technical Specification Group meeting in Phoenix. All the major test equipment suppliers involved, including Anite, Anritsu, Racal, Rohde & Schwarz and Spirent, have submitted test cases for final approval, and observers believe that should boost confidence in the testing process.
"Yes, we are taking the flack, and it has taken longer to sort this out than envisaged," said Richard Catmur, European business development director at Spirent Communications. "But I, or the rest of the industry, can't think of a better way of doing this so that it is done strictly and in the long-term interests of the mobile sector."
Spirent writes test cases and supplies equipment for testing the RF aspect of handsets, and has a reciprocal partnership with Anite on the protocol-conformance testing. "We had problems getting hold of handsets," Catmur said. "It all comes down to the competition between the suppliers. And then there is a succession of small and irritating issues related to GCF. They can get overly picky regarding the paperwork and perhaps should be less bureaucratic in their approach."
For the RF side, one of the biggest outstanding issues is Section 8 of the 3GPP's TS 34.121 specs--specifically, the Radio Resource Management. Catmur says tests for this item have not been finalized, so test cases cannot be written, which is holding up progress.
The most contentious issue and biggest potential obstacle to a successful rollout of 3G is intergeneration handover testing. "Handover compliance and testing on 3G handsets will be mandatory from late 2004, but operators are demanding phones that hand over seamlessly now between GSM and UMTS," Catmur said. "We really need to respond to this positively, and put more emphasis on the test cases to achieve this."
At Anritsu, which has had test cases approved for RF and protocol testing, principal technologist Dan Fox says a huge number of things need to happen in parallel in the conformance-testing process. "But realistically, we can only progress at the pace of the slowest contributor--and at the moment, that is the handset makers," said Fox. <<
###
From 3GPP TSG#22 Highlights (December 2003) for TSG-T (Terminals)
Testing:
Progress was reported on the area of Tree & Tabular Combined Notation (TTCN) test case development. TTCN has now moved to March 2003 baseline. 83 Package 1 (P1) test cases verified by WG T1 to March 2003, 22 Package 2 test cases verified by WG T1 to March 2003 baseline. Major effort by industry to re-verify all test cases to new baseline completed in late October. Approved test cases will be published in version 3.4.0 of TS 34.123-3. Motorola has verified internally 92 P1, 74 P2 and 43 P3 test cases against the September 2002 base line which has given rise to many corrections to the ‘live’ suite.
The need was identified by WG T1 for a specific release and configuration manager within MCC Task 160. The Global Certification Forum (GCF) offered to co-fund the post (up to the equivalent of 6 man-months). The remaining funding is to be found from the allocation of 58 man-months already approved for MCC Task 160. TSG-T endorsed the creation of this post.
In order to increase the TTCN maintenance capabilities and change traceability, MCC Task 160 reported that they installed a chain of TTCN-2 tools for editing, analysing/compiling, merging and comparisons, the configuration management tool, and a series of working procedures for TTCN configuration management, error handling /corrections and deliveries.
Based on the WG T1 recommendations, all 3GPP funding for 2004 will be allocated to Frequency Division Duplex (FDD) TTCN (details in the STF 160 report TP-030195). Target is to complete all GCF high priority by late 2004.
The proportion of complete Radio Resource Management (RRM) tests is approximately 24%. Completion of the outstanding RRM test cases is required within 9 months for subsequent validation/ approval by GCF before the end of 2004. The RF Convener is putting into place a strategic plan to cover the work but more offers of assistance are always welcome.
CRs to TS 34.121 introduced testing for the 800MHz band were approved. Rapid contributions were made in time for approval at T#22 in order to meet Japanese Regulatory conditions for the introduction of DS CDMA for the testing of band VI frequencies.
A single Release of TS 34.122 has been prepared for approval as a result of a T1 study into the best method of a maintaining a single release document based on Release-5 to cover Release '99, Release-4 and Release-5. It has been made clear to the customer where the specifications refer to the different core specification releases. Updated versions of the Release '99 & Release-4 docs (for approval) contain pointers to the new (Release-5) Single Release specification.
Problems of potential misalignment of inter Radio Access technology (RAT) test cases between WG T1 and TSG GERAN are being addressed. GERAN has responsibility for GERAN to UTRAN handover prose tests but T1/ETSI MCC 160 is responsible for the TTCN development. GERAN is reviewing its approval process and WG T1 will liaise closely to provide assistance as required. <<
The quarterly plenary, TSG#23, was held last week in Phoenix, AZ, so an update will follow.
- Eric - |
