Large-scale projects can often be a hectic endeavor for engineering. As these engagements splinter out into less familiar hardware and platforms, the development and testing can get increasingly confusing. We often end up dealing with languages and frameworks that are as unfamiliar as the associated hardware and platforms, increasing the focus and time required to ensure that all versions of the application are of the highest quality. Under normal circumstances, engineers could lean heavily on test automation. With the test automation doing the majority of the repetitive tasks, test engineers have more time to focus on other aspects of testing.
Unfortunately, Roku stuck out as something that was never able to be automated. Roku Channels, written in BrightScript and without any automation support, often became challenging for engineering teams. Test Engineers were forced to execute all tests manually in order to validate builds. But all of that changed on December 9th, 2019 when Roku announced and released their own test automation software! Their automation uses the widely popular Selenium framework and aims to help developers and test engineers expedite the development lifecycle of Roku channels.
How Does Roku Test Automation Work?
While the framework uses Selenium, it doesn’t use it in the way that most people would immediately assume. It uses Selenium’s JSON Wire Protocol, which means that in order to interact with the Roku device, there must be a RESTful web service to receive JSON over HTTP. This is the underlying framework that allows Selenium (and Appium) to communicate between the written automation code and the various WebDrivers. Following the naming convention from Selenium, Roku’s custom web service is simply called the Roku WebDriver.
What Makes Roku Test Automation Different?
The first major difference between Roku test automation and more traditional test automation is the supported language. The automation is based on the JSON Wire Protocol, which means that any language that can make HTTP requests can be used to for Roku automation. However, the initial release only has the HTTP requests easily accessible and fleshed out via the Roku Robot Framework.
The Roku Robot Framework uses the Robot Framework, which itself is built on Python. There are two Python files that do the heavy lifting. The first,
webDriver.py is the core of the interaction between JSON Wire Protocol and the WebDriver. The functions within the WebDriver class are used to create the HTTP requests using the
api.py module to implement Python’s built in request functionality. The second file,
RobotLibrary.py, instantiates a WebDriver, and then uses the HTTP requests provided by the WebDriver to execute a series of commands which cause automated interaction with the Roku device. Because most functions in
RobotLibrary.py are tagged with custom keywords, when calling these functions in a
.robot test file, the functions can be referenced in a way resembling natural language. For example, instead of having to call the
launchTheChannel() function by name, we can instead reference the keyword and execute the function by saying “Launch the Channel”.
The second major difference between Roku test automation and more traditional test automation is the lack of any sort of virtualization. Roku automation requires that the WebDriver be able to communicate directly with a physical Roku device. This is most easily achieved by having the automation code, WebDriver, and Roku device all on the same network. In order to get new builds of a channel onto the Roku Device, the channel must be sideloaded manually, and Roku restricts devices to only having one sideloaded channel on a device at a time. All of the above differences can cause frustrations in development and testing, and unfortunately leave few to no options for integrating automated tests into a CI pipeline. My best suggestion is to have engineers run the automated tests prior to creating their pull requests. While this may leave much to be desired, it is still better than having no way to automate tests.
How Can Roku Test Automation Be Implemented?
Roku Test Automation can easily be implemented following the instructions given here. The instructions are straightforward and thorough, along with examples of sample tests that can be run. If you are unfamiliar with the Robot Framework, there is a lot of documentation on Robot Framework, the Roku Framework Library, and the Roku WebDriver. We’re still working on a strategy to efficiently import the above framework for future projects, but at this moment we’re unable to add the Roku Robot Framework into a test automation suite without copying and pasting the provided python files. The appeal of implementing a test framework that uses the Roku Robot Framework directly is that tests could be written to use the Python functions provided by Roku. This approach theoretically should offer greater support from Roku, as well as reduced overhead in having to develop and maintain the functionality that can interact with the WebDriver.
With Roku’s announcement of their test automation framework, they’ve opened up automation more than it has ever been on Roku devices. While there are still some things left to be desired in terms of automation support, such as a framework in a more well-known language or an emulator to allow for CI automated testing, the impact of this announcement is huge for automation practices around Roku and should allow for a more expedited development lifecycle for Roku channels.