Connections +

Testing Through the Air

If you are going to deploy wireless technology, you need a test plan. If in doubt, have a specialist review your strategy

April 1, 2004  

Print this page

The last two years have seen a big push by the telecom industry to accelerate the use of wireless. New technologies abound, including WAP, i-mode, J2Me, VoiceXML, Bluetooth, GPRS, MMS and Wireless LAN’s, to name but a few.

Slowly, but surely they will start to impact on how we do business, thus setting new challenges for the IT functions.

The biggest problem is proving wireless applications will work. Wireless networks can show up all sorts of application problems, which remained undetected in the development shop.

Major challenges

In performing testing of the traditional distributed application, it is fairly easy to re-create the production environment. There are usually client PCs, corporate servers and an intranet or extranet based on Internet protocols; all equipment which can be duplicated at least in part.

Even if we can’t prove the full scalability of the application, we can have a pretty good idea that the application will at least work when we deploy it.

It’s different with wireless. We can’t recreate the environment as easily because we don’t have a wireless infrastructure in our test lab. Even if we can simulate some aspects of the application — the handset, for example — we can’t really be sure what happens when we try it over a real wireless network.

It’s a good idea to get some wireless testing into your launch plans. If you can’t test over the existing live networks, talk to the service providers as many of them have test networks and may be able to help you gain a bit more confidence before you go live.


Mobile telephony has always created the fear of someone hacking in and listening. The data is out there “in the air” somewhere and has that feel of vulnerability. Although it’s now possible to buy goods via your phone, not many people are taking up that challenge.

Today, even Internet security is still a developing area, but it is possible for a business to implement client-server technology, which will give a high level of confidence using public/private encryption keys.

To implement secure transactions in a mobile network requires support from the underlying network provider to embed the encryption technology in the service. Thus, you are depending more here on the service provider.

While you can take the view that the testing is the service provider’s responsibility, it makes sense to test security carefully. After all, any security failure will reflect more on your system than it will on the underlying network operator.

To get access to the data highways in a corporate wired network is difficult providing the external Internet access is properly controlled via firewalls. A determined hacker, however, could pick up the Wireless LAN.

Armed with a laptop with a wireless LAN card, the hacker could pull up in the car park next door to your facility and “listen” in to the traffic on your network and then use the data to get inside. While the Wireless LAN provides a built-in security feature (Wired Equivalent Privacy or WEP)), recent research has shown that WEP may be vulnerable to external attacks and should be augmented by other security methods.

Of course it’s possible to have secure Wireless LAN but the need is there for increased security testing. And the problem is that mobile security is less well understood and your internal IT department or your external firewall testers may not have the capability to test this.

At the current time, people who understand both wireless and security are thin on the ground and you would be well advised to consider external specialist help before you declare your wireless application to be fully security tested.

Mobile features

So far we have focussed on testing of mobile functionality in areas where mobile is providing the equivalent functionality to that of a client-server or fixed wire Internet environment.

The major benefit of mobile technology is just that — we can locate somebody’s whereabouts and build applications that take advantage of that information.

The obvious example is directories of information giving the location of the nearest pub or restaurant, but more sophisticated examples are services which locate both you and other moving objects — how long do I have to wait at this stop before the next bus arrives?

Testing this will be tricky as not only do you have to check out the functionality of the application, but the accuracy of the information has to be verified and tested frequently if the application is to meet user expectations.

Location-based services also represent a security challenge. Knowledge of your whereabouts is considered highly confidential information and will be subject to data protection legislation in many countries.

The need to protect this information will be paramount if companies are to avoid financial penalties from government agencies.

Thus, more sophisticated forms of penetration testing will be required to prove that location-based information is properly protected from external hackers.

With mobile we have an explosion of devices all with different characteristics, which means that testing on one device does not guarantee working on another.

It isn’t going to be possible to test your application on all devices. Similarly, differences can exist between the different networks.

Early adopters have tended to restrict access to their services to one or two handsets with the service available through perhaps only one network. This is a pragmatic approach, but to get more consumers using these services, choice of device and network is essential.

One approach to solving this problem is NCC Group’s mCheck tool. This tool has a database of device and network profiles. It can navigate a mobile application site and report back on problems based on its knowledge of mobile phones, PDA’s etc.

The tool can pick up obvious problems such as broken links but will also find some very subtle issues such as use of card titles (some handsets don’t display these so if your application uses them your customer may never see important data or instructions).

Although you can never totally eliminate some manual testing, tools such as these can save large amounts of effort both in testing and detecting problems, which would be costly to fix once an application goes live.

WAP Case study

If ever there was an example of why testing is so important, the introduction of the Wireless Application Protocol (WAP) has highlighted the case.

It has been said that, for any WAP application to be useful, it has to do one of two things: save time, or kill time. Within this somewhat broad definition lie many thousands of WAP sites, offering us the tantalising prospect of transacting our business anytime, anyplace and anywhere.

The problem is, when it comes to the phone in your pocket, many of them disappoint in a big way.

One of the biggest areas of failure lies in the lack of focus on a well-constructed test and interoperability environment.

Many of the common problems that mar users’ experiences of the wireless Internet are avoidable by the use of current test services and tools. By way of example, here are some of failures encountered by NCC Group when testing wireless sites and gateways over the past 12 months:

Sites, which cannot scale for large numbers of users;

Reliance on card titles and fonts (which some phones do not support) to convey important information;

Getting ‘trapped’ in a card where the only way out is to call the browser’s main menu or drop the connection;

Inconsistent navigation structures;

Displaying outdated information;

Displaying improper content without warning;

Inaccurate links (mainly external links), and

Fundamental lack of interoperability between gateways and handsets.

Most of these problems could have been avoided if the developers had given more attention to testing. The industry is responding to these problems by setting up testing and certification schemes or providing test lab environments for developers to debug in.

Mobile applications are a new industry and call for new skills and techniques w
hen testing them. Much of these skills are found within the vendor and network provider, but they are still scarce in the traditional testing industry.

There are many good companies out there with good testing skills for distributed and Web-based applications, but many of these lack the telecom experience to fully appreciate the wireless testing issues.

If you are going to deploy wireless technology in your business, then plan your testing early and if in doubt, have a specialist review your strategy and supplement the skills of your existing test team.

The potential for wireless technology applications is an immensely compelling one for business and consumers alike. Success is guaranteed for those who can deliver on this promise, but failure is very public.

It will take far-sighted people who can understand the importance of robust, quality services using handsets and networks that can communicate well.

And that means treating the testing process with the same importance as the development process.

Jane Pink is Director of Specialist Testing Solutions at NCC Group in Manchester, England. NCC Group is the country’s leading independent provider of IT Assurance, Security and Advice.



Canada’s Wi-Fi industry has received a major boost from national wireless carriers following the recent signing of an inter-carrier agreement that establishes common standards for roaming and interoperability of the public hotspots they operate.

In addition, Bell Mobility, Microcell Solutions, Rogers AT&T Wireless and Telus Mobility plan to create a common brand identifier for Wi-Fi hotspots across the country.

Under terms of the agreement, all public commercial hotspots operated by the carriers, and any other Canadian operator or hotspot owner who meets the minimum requirements and chooses to join the roaming alliance, will be branded consistently with the common hotspot identifier.

When customers access Wi-Fi service in any location with the hotspot identifier, they will be presented with an identical, browser-based login area.

“Increasing the number of locations and providing consistency of service will eliminate two of the biggest barriers to Wi-Fi gaining mainstream popularity,” said Peter Barnes, President and CEO of the Canadian Wireless Telecommunications Association (CWTA).

In an effort to spur the availability of public commercial hotspots, the four carriers have committed to build more than 500 hotspot locations over the next year.

Roaming between the four wireless carriers’ hotspots is expected to be available this fall.

Meanwhile, BOLDstreet Wireless Internet released its Global Roaming Service Platform (GRSP) in March designed to increase Wi-Fi usage and Wi-Fi-enabled applications.

The GRSP, which allows corporate users and consumers to securely access thousands of hotspots worldwide, offers simplified billing through mobile phones, prepaid services and SMS options.

The extended hotspot service, along with the new billing options, is intended to give carriers and ISPs more opportunities to generate revenue from their subscriber base and to attract new customers.

BOLDstreet will deploy the platform globally, beginning in North America in the next quarter. Nomadix Inc., a supplier of public access solutions, is providing the global access equipment.

Print this page