February 19, 2020
Quality Engineering

IoT Testing: The Things To Know To Do It Right

IoT Testing: The Things To Know To Do It Right

Vlada Benyukh, Agile Delivery Lead of Quality Engineering Practices. Vlada is ISTQB, ICAgile and SAFe certified Agile Delivery Leader of the Quality Engineering Services. She has 9 years of experience in testing of Web/Mobile-based systems for Finance, Betting, eCommerce, Travel and IoT specific projects. Vlada is heading a group of up to 30 people.

Across the globe, digital technology is connecting people in more interesting ways than ever before. Kickstarted by the personal computer revolution of the 1970s, users became accustomed to using computing devices to simplify tasks and bring new ideas to life. The introduction of the internet propelled our connected world, linking devices together from any point on Earth to share information at light speed. And after the turn of the century, the smartphone’s debut gave users around-the-clock access to instant messaging and internet connectivity through a portable device that lives in the pocket, keeping billions of people linked together at any given moment.  

One of the greatest evolutionary steps forward in connected technology is the Internet-of-Things (IoT). Made possible through the miniaturization of connected wireless sensors, power-efficient battery life and a vast, global manufacturing supply chain, IoT devices are ushering in a new era of always-connected intelligent capabilities. From smart speaker devices like Amazon Alexa to small package sensors to track warehouse inventory, the IoT helps gather and analyse vast amounts of data, leading to better decision making, more responsive devices and a world that stays effortless in touch. 

To take full advantage of new capabilities made possible by the IoT, developers must be prepared to build solutions that are easy-to-use, intuitive, reliable and secure.  Whether it’s a consumer product or an industrial workhorse, devices must be rigorously tested to ensure they’re not draining power, leaking data or so challenging to operate to the point of uselessness. Of course, because IoT devices are often small, take unique form factors and can operate continuously without interruption, testing for flaws and usability can present its own set of challenges. 

Here are the most crucial IoT tests that must be performed, as well as a number of remedies to make testing tasks run smoothly:

Usability testing

What is the point of building a device if it isn’t useful? That may seem like a simple question, but when hardware and software companies build solutions with capabilities hidden behind messy user interfaces or complicated series of button presses, it can seem like the developer never came up with a fundamental answer.

In the case of Shape Robotics, creator of the Fable modular educational robot, the device required a companion iPad app easy enough to be operated by children. In addition to ensuring basic functionality worked, the app required extensive testing to ensure the software was intuitive enough for young users. For Shape Robotics, this meant testing from the user’s point of view — which, in this instance, was an audience of primary school students.

QA consultants must test their devices from the point of view of the end-user. Essential functionality, notifications and data synchronisation should be presented to the user in the clearest, simplest terms possible, ensuring the user can understand exactly what they’re trying to do with their device or accompanying software. 

End-to-end product QA testing (including upgrades/updates of firmware/hardware parts)

When it’s time to update an IoT device, the upgrade can often come in multiple parts: one, an app update, and two, a firmware update. Both updates are important, but firmware upgrades, in particular, can be critical to the security or functionality of an IoT device. But a firmware update that’s not ready for prime time can end up crippling an IoT device, requiring significant user intervention to ultimately remediate the problem.

Before releasing any software or firmware updates to users, ensure the updates have been thoroughly tested across a wide variety of devices. This can be achieved through IoT test automation, which can result in faster releases at lower costs. 

Security testing

The end-user uses a mobile app and Bluetooth to provision the WiFi settings on the device.  And after that, the interaction between the mobile and the hardware part is via the cloud. Security testing should occur with Cloud APIs, Network communication, the same as Hardware examination and Hardware attacks should be checked, because ensuring your IoT product is secure is of the utmost importance. This isn’t a simple matter of adding encryption at the application layer and calling it a day. Testing to ensure strong IoT security requires a holistic approach from authentication to use, ensuring devices are safe and secure enough to operate. As far as it was mentioned above, all interaction between the mobile and the hardware part is via the cloud APIs, the private or sensitive information may pass through an IoT device, securing its data integrity is necessary to ensure it isn’t being improperly stored or leaked. Check the use of public and private APIs to identify any potential red flags.  

Interoperability testing

One of the beauties of the IoT is its wide range of connectivity. Many IoT devices support various different operating systems, mobile devices, hardware, firmware etc. This, of course, means IoT devices must be tested with as many other devices as possible to ensure interoperability. Testing devices for interoperability ensure that products work together securely, without sacrificing performance, all data transferred properly from one device to other platforms. Sometimes for such kind of checks, it is crucial to use emulations of different actions. One solution that may connect with 99% of devices could encounter critical flaws with the remaining 1%.

Reliability testing

The proliferation of many IoT devices has been spurred by reliability offered through connected technology. Many IoT devices are expected to operate around-the-clock without interference. In order to live up to this expectation, IoT devices must be tested in a simulated, real-world environment. Virtualised sensors and equipment can accurately portray how your device will operate in the wild — and highlight any critical flaws before being released to users. 

Real-time testing

Because many IoT devices are relied on for immediate feedback, it’s also necessary to test for real-time communications capabilities. Again, this can be achieved through virtualised environments that simulate transfer speeds, data quality and overall responsiveness. 

Performance testing

IoT devices often require high performance from as few resources as possible, with many devices being comprised of little more than a wireless radio, a system-on-a-chip and a battery. Even though engineering marvels have shrunk functional computers down to the size of a coin, it’s a developer’s responsibility to ensure their software or firmware isn’t bloated or overly complex. Testing for the speed, responsiveness and quality of the underlying code can ensure an IoT device’s priceless limited power isn’t overloaded. 

Connectivity testing

Because IoT devices must often connect to a wide variety of wireless networks or other devices, ensuring connectivity is strong, fast and secure is crucial. Whether it’s over Bluetooth, WiFi, Ethernet, a physical cable or anything else in between, be sure the IoT device is able to relay information successfully.

Regulatory testing

Many IoT devices operate in particular industries with strict government regulations, like medicine or transportation. Take care to ensure IoT devices meet all rigorous regulatory standards in order to avoid any costly fines or penalties.

Read also: