Mobile Automation - Creating scalable device farms using open source technologies

 Introduction:

In our consumer driven society, with the leverage of advanced technologies, businesses are in cut-throat competition with growing demands of faster releases of contents, apps and software patches. In the era of continuous integration, continuous delivery, automation and AI, it is of paramount interest for the businesses to ensure "continuous quality" in their releases. Since it is not possible for humans to verify and validate everything simultaneously in a coherent fashion, we are heavily dependent on test automation.

Some pressing challenges in Mobile automation:

• Purchasing physical mobile devices -> it will lead to capital expenditure
• Mobile test automation tools -> they are excellent, but incurs license costs which perhaps impact the operational expenditure. Not only that, businesses often needs to pay additionally for professional support. Often businesses need to wait to get any new feature added in the tool exclusively to support their needs.
• Using Cloud based device farms -> they are excellent, but using cloud based device farms may not reduce the operational expenses of a business. For large businesses, it may offer a trade off but for smaller businesses it may be costlier.
• Outsourcing -> a brilliant idea when the businesses are not having any internal IT teams, or they do not have existing capability, or they want to externalize the delivery risk.

Why to use open source technologies for Mobile Automation?

Well, if you have read the introduction, by now you have a clear idea of why to use open source technologies. The most crucial factors are:

• it comes free of cost
• supports a wide array of mobiles OS (Android & iOS) and devices to a great extent
• with right skills it is easy to set up. Using open source technologies are not a "rocket" science, so whoever is really interested in upskilling themselves, they can learn it quickly
• using containerization, it is scalable to an enterprise grade. But that will incur good amount of infrastructure and specialized personnel costs (i.e. Security Specialists, DevOps engineers)
• the open source communities are very active and provides a lot of help and support whenever needed

Architectural overview of our first solution - using device emulator running on host machine / connecting physical devices

In our present architecture, we are going to run Appium server inside Docker Desktop, while our device emulator will be running in the host machine. So, we need to use TCPIP to establish connection between the Appium Server container's ADB and our host machines device emulator ADB. 

image courtesy: Gemini AI

Remember, you can also connect your physical android devices to the Host Machine using USB and use that devices UDID to run tests directly on that device.

If you want to scale up execution in multiple physical devices, then you can create multiple copies of your test scripts. In each scripts, expose unique Appium Server, System Port and mpegServerPort,  and update the unique UDID of the connected devices.

Infrastructure needs and software installations

First, you need to ensure that you have the right infrastructure available. The machine you are using should have at least 8 GB RAM, 5 to 12 core CPU, at least 20 - 50 GB of free Disk Space, a good graphics card (optional in Windows machine), high speed internet connection. Windows 10/11 users, you need to ensure you have WSL2 installed and enabled virtualization. Linux/MacOS, users to concentrate on Virtual Machine Platform and KVM acceleration following available instructions over the internet. 

Second, you need to install the required softwares:

1. Docker Desktop / Podman (In this article, I am covering only Docker Desktop)
2. Android Studio - latest version is recommended
3. IntelliJ / Eclipse
4. Open JDK 17.0 or Open JDK 21.0 (Remember to set JAVA_HOME and adding it to Environment Variable)

Third, verify installed softwares quickly by doing the following:

• Run CMD as Administrator, type "java --version" -> Ensure it returns correct java version
• In the same CMD window type "docker --version" -> Ensure it returns correct docker version
• In the same CMD window type "docker ps" -> Ensure the command does not return any errors
• Open IntelliJ, ensure it is opening. Then go to its terminal window and type "java --version" -> Ensure it is returning right java version

• Open Android Studio and verify that you are able to see "More Actions" link
• Click on it, and in the drop down menu you will see "Virtual Device Manager"
• Click on it, and it should open "Device Manager" window

Creating virtual device emulator and ADB set up

Now you are ready to begin your adventure. In this article we will deal with Virtual Mobile Emulator devices only in Windows 11, but the set up will work on real mobile devices if you connect it to your system using USB.

You can create a virtual emulator device from the Device Emulator window, you may have noticed in the previous screenshot that I have created a device named "Small Phone". Once you create the device, run it and within a minute or two you will see the device emulator window in your screen. I am not covering it in detail, you can find guides on how to create emulator devices in the internet.

Run CMD as administeator, navigate to this path: c:/users/<your user profile>/appData/local/android/sdk/platform-tools

Execute this commad: 

adb devices

Your emulator device should get listed as follows:

In the CMD window, execute below commands:

1. adb tcpip 5555

2. adb connect localhost:5555

3. adb devices

Creating a Java project for mobile automation

You can create a Java Test project in IntelliJ or in your choice of IDE using either Gradle or Maven as a build tool. We are going to use TestContainers in Java to dynamically run the Appium container (Appium Server) as part of our test script. Add below dependencies (in this example I am using Maven):

<dependencies>

<dependency>

<groupId>org.testcontainers</groupId>

<artifactId>testcontainers</artifactId>

<version>2.0.5</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>com.github.docker-java</groupId>

<artifactId>docker-java</artifactId>

<version>3.7.1</version>

<scope>compile</scope>

</dependency>

<!-- Source: https://mvnrepository.com/artifact/io.appium/java-client -->

<dependency>

<groupId>io.appium</groupId>

<artifactId>java-client</artifactId>

<version>10.1.1</version>

<scope>test</scope>

</dependency>

</dependencies>

You can refer the below class file code which runs the test in the emulator device using appium server container in the host machine. You must focus on the code that creates the appium container and the UiAutomator2 capabilities that I have used. You may not have come across any code so far where everything is kept under the same hood for easy understanding of Appium container's environment variables and the needed capabilities of UiAutomator2.

Remember:

1.  Copying the adbkey and adbkey.pub files to the container is necessary. You can access these files in your machine under c:/users/<your user profile>/.android folder.
2. Run Docker Desktop engine in your host machine before running the test class
3. Ensure your device emulator is running in your host machine

*********************************************************************

import io.appium.java_client.android.AndroidDriver;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.testcontainers.Testcontainers;

import org.testcontainers.containers.GenericContainer;

import org.testcontainers.containers.wait.strategy.Wait;

import org.testcontainers.utility.MountableFile;

import java.io.IOException;

import java.net.MalformedURLException;

import java.net.URL;

import java.time.Duration;

public class MobileTrial {

    public static void main(String[] args) throws IOException, InterruptedException {

        Testcontainers.exposeHostPorts(5555, 5037, 8200);

        GenericContainer<?> appiumServerContainer = new GenericContainer<>("appium/appium:latest")

                .withExposedPorts(4723,8200)

                .withPrivilegedMode(true)

                .withAccessToHost(true)

                .withEnv("REMOTE_ADB", "true")

                .withEnv("ANDROID_DEVICES", "host.testcontainers.internal:5555")

                .withEnv("APPIUM_ADDITIONAL_PARAMS", "--relaxed-security --allow-insecure=*:chromedriver_autodownload")

                .withCopyFileToContainer(MountableFile.forHostPath("c:/users/<put your user directory>/.android/adbkey"), "/home/androidusr/.android/adbkey")

                .withCopyFileToContainer(MountableFile.forHostPath("c:/users/<put your user directory>/.android/adbkey.pub"), "/home/androidusr/.android/adbkey.pub")

                .withCopyFileToContainer(MountableFile.forHostPath("c:/users/<put your user directory>/.android/adbkey"), "/root/.android/adbkey")

                .withCopyFileToContainer(MountableFile.forHostPath("c:/users/<put your user directory>/.android/adbkey.pub"), "/root/.android/adbkey.pub")

                .waitingFor(Wait.forHttp("/status").forPort(4723).withStartupTimeout(Duration.ofMinutes(2)));

        appiumServerContainer.start();

        String appium_server_url = "http://" + appiumServerContainer.getHost() + ":" + appiumServerContainer.getMappedPort(4723);

        System.out.println("Appium server url: " + appium_server_url+"/status");

        DesiredCapabilities caps = new DesiredCapabilities();

        caps.setCapability("appium:automationName", "UiAutomator2");

        caps.setCapability("platformName", "Android");

        caps.setCapability("browserName", "Chrome");

        caps.setCapability("appium:deviceName", "Android Emulator");

        caps.setCapability("appium:udid", "host.testcontainers.internal:5555");

        caps.setCapability("appium:platformVersion", "15");

        caps.setCapability("appium:chromedriverConnectHost", "127.0.0.1");

        caps.setCapability("appium:chromedriverConnectPort", 8200);

        caps.setCapability("appium:systemPort", 8200);

        caps.setCapability("appium:automatedChromedriverDownload", true);

        caps.setCapability("appium:ignoreHiddenApiPolicyError", true);

        caps.setCapability("appium:relaxedSecurity", true);

        caps.setCapability("appium:noSign", true);

        caps.setCapability("appium:amStartWithCleanRules", true);

        caps.setCapability("appium:noReset", true);

        caps.setCapability("appium:clearDeviceLogsOnStart", true);

        caps.setCapability("appium:skipUnlock", true);

        caps.setCapability("appium:appWaitDuration", 60000);

        caps.setCapability("appium:suppressKillServer", true);

        caps.setCapability("appium:enforceAppInstall", true);

        caps.setCapability("appium:uninstallAppiumServer", true);

        caps.setCapability("appium:uiautomator2ServerLaunchTimeout", 60000);

        try {

            AndroidDriver driver = new AndroidDriver(new URL(appium_server_url), caps);

            driver.get("https://www.saucedemo.com/");

            System.out.println(driver.getTitle());

            driver.quit();

        }catch(Exception e){

            System.out.println(appiumServerContainer.getLogs());

        }

    }

}

*********************************************************************************

Architectural overview of our second solution - using device emulators running in Test Containers

Coming soon...