DroidFarm Testing Guide¶

This guide explains how to write, run, and integrate Appium tests against DroidFarm-managed Android virtual devices.

1. How DroidFarm Connects to Appium¶

flowchart LR
    subgraph K8s["Kubernetes Cluster"]
        subgraph Pool["DevicePool (staging-pool)"]
            subgraph Pod["Cuttlefish pod (pool-emulator-0)"]
                Android["Android guest VM"]
                Appium["Appium Server\n:4723"]
                Android -->|"ADB"| Appium
            end
        end
        Session["TestSession\nstatus.appiumEndpoint\nstatus.streamURL"]
    end

    CI["CI runner / pytest"]
    Browser["Browser preview"]

    Appium <-->|"WebDriver\nhttp://pool-emulator-0\n.droidfarm-system.svc:4723"| CI
    Session -->|"appiumEndpoint"| CI
    Session -->|"streamURL"| Browser

Flow:

1. You (or CI) create a TestSession resource pointing at a DevicePool.
2. The DroidFarm operator claims a Cuttlefish pod from the pool and transitions the session through Pending → Claiming → Preparing → Running.
3. Once Running, status.appiumEndpoint contains the WebDriver URL your tests connect to.
4. Your test process (pytest, a CI job, a test container) sends W3C WebDriver commands to that URL. The Appium sidecar inside the pod translates them to ADB calls against the running Android guest.
5. Patch status.result to Pass, Fail, or Error. The operator transitions through Collecting (where artifacts are recorded) then to the terminal state. The device is released back to the pool.

2. Writing Tests — Minimal Example¶

import os
from appium import webdriver
from appium.options import UiAutomator2Options
from appium.webdriver.common.appiumby import AppiumBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

APPIUM_URL = os.environ["APPIUM_URL"]   # from status.appiumEndpoint

options = UiAutomator2Options()
options.platform_name = "Android"
options.automation_name = "UiAutomator2"
options.app_package = "com.example.myapp"
options.app_activity = "com.example.myapp.MainActivity"
options.no_reset = False          # clear app data before session
options.new_command_timeout = 300

driver = webdriver.Remote(command_executor=APPIUM_URL, options=options)

try:
    wait = WebDriverWait(driver, 30)
    title = wait.until(
        EC.visibility_of_element_located((AppiumBy.ID, "com.example.myapp:id/tv_main_title"))
    )
    assert "Welcome" in title.text
finally:
    driver.quit()

Install dependencies:

pip install Appium-Python-Client>=3.1.0 pytest>=7.4.0

The full example suite lives in examples/tests/python/.

3. Running Tests Locally Against DroidFarm¶

Prerequisites¶

• kubectl configured to reach the cluster
• A DevicePool with at least one idle device

Step-by-step¶

# 1. Create a TestSession and capture its name.
SESSION=$(kubectl create \
  -f examples/ci/testsession-ci.yaml \
  --namespace droidfarm-system \
  -o jsonpath='{.metadata.name}')
echo "Session: $SESSION"

# 2. Wait for it to become Running.
kubectl wait testsession "$SESSION" \
  --namespace droidfarm-system \
  --for=jsonpath='{.status.phase}'=Running \
  --timeout=300s

# 3. Extract the Appium endpoint.
APPIUM_URL=$(kubectl get testsession "$SESSION" \
  --namespace droidfarm-system \
  -o jsonpath='{.status.appiumEndpoint}')
echo "Appium: $APPIUM_URL"

# 4. Optionally open the live stream in a browser.
STREAM_URL=$(kubectl get testsession "$SESSION" \
  --namespace droidfarm-system \
  -o jsonpath='{.status.streamURL}')
open "$STREAM_URL"   # macOS; use xdg-open on Linux

# 5. Install dependencies and run tests.
cd examples/tests/python
pip install -r requirements.txt
APPIUM_URL="$APPIUM_URL" pytest . -v --tb=short

# 6. Clean up — patch to Succeeded so the device is returned to the pool.
kubectl patch testsession "$SESSION" \
  --namespace droidfarm-system \
  --type=merge \
  --patch '{"status":{"phase":"Succeeded","result":"Pass"}}' \
  --subresource=status

Targeting a specific test¶

APPIUM_URL="$APPIUM_URL" pytest test_app_launch.py::TestAppLaunch::test_app_launches_successfully -v

Parallel execution with pytest-xdist¶

Each worker needs its own device, so create multiple TestSessions first (one per worker), then pass the endpoints via environment variables and the -n flag:

APPIUM_URL="$APPIUM_URL_1" pytest -n 2 examples/tests/python/ --dist=loadfile

Note: --dist=loadfile ensures that tests in the same file run on the same worker (important for stateful tests like test_clear_data_between_runs).

4. CI Integration¶

The reference GitHub Actions workflow is at examples/ci/github-actions-test.yaml.

Required secrets¶

Workflow summary¶

1. Create TestSession — kubectl create -f examples/ci/testsession-ci.yaml
2. Poll until Running — checks status.phase every 10 seconds (10-minute ceiling)
3. Extract endpoints — reads status.appiumEndpoint and status.streamURL
4. Run pytest — APPIUM_URL=<endpoint> pytest examples/tests/python/ -v --tb=short --junitxml=test-results/junit.xml
5. Patch session — sets status.phase to Succeeded or Failed
6. Upload to GitHub Actions — JUnit XML results
7. Comment on PR — posts a Markdown table with pass/fail counts

5. Writing Stable Mobile Tests¶

Flakiness in mobile UI tests is almost always caused by timing assumptions. Follow these rules to keep your test suite reliable.

Always use explicit waits¶

# BAD — implicit wait races the rendering pipeline
element = driver.find_element(AppiumBy.ID, "com.example.myapp:id/btn_submit")

# GOOD — wait until element is visible before interacting
wait = WebDriverWait(driver, 30)
element = wait.until(
    EC.visibility_of_element_located((AppiumBy.ID, "com.example.myapp:id/btn_submit"))
)
element.click()

Prefer resource-id locators over XPath text¶

# FRAGILE — breaks when translations change
driver.find_element(AppiumBy.XPATH, "//android.widget.Button[@text='Submit']")

# STABLE — resource IDs are defined in code, not translations
driver.find_element(AppiumBy.ID, "com.example.myapp:id/btn_submit")

Wrap network-dependent actions in a retry loop¶

API calls made by the app can be slow on first launch. Build a small helper:

import time
from selenium.common.exceptions import TimeoutException

def wait_for_with_retry(driver, by, locator, retries=3, timeout=20):
    for attempt in range(retries):
        try:
            return WebDriverWait(driver, timeout).until(
                EC.presence_of_element_located((by, locator))
            )
        except TimeoutException:
            if attempt == retries - 1:
                raise
            driver.back()   # dismiss any blocking dialogs
    raise TimeoutException(f"Element {locator} not found after {retries} attempts")

Use driver.background_app instead of time.sleep¶

# BAD
time.sleep(5)

# GOOD — send app to background for a known duration and resume
driver.background_app(3)

Keep each test independent¶

Every test should be able to run in isolation. Never share state through class-level variables or file-system writes; always navigate back to the main screen at the start of each test (or rely on noReset: False + autoLaunch to restart the app).

Use pytest.mark.flaky sparingly¶

The pytest-rerunfailures plugin lets you rerun flaky tests:

pip install pytest-rerunfailures
pytest --reruns 2 --reruns-delay 5

Reserve retries for genuinely non-deterministic external dependencies (e.g., network calls to a staging backend), not for timing issues in your test code.

6. Test Isolation — clearDataBetweenSessions and recycleAfterSessions¶

DroidFarm provides two isolation knobs in the DeviceTemplate:

clearDataBetweenSessions: true¶

appConfig:
  - packageName: com.example.myapp
    clearDataBetweenSessions: true

The operator runs adb shell pm clear com.example.myapp during the Preparing phase, before the Appium session is opened. Effect:

• App's SharedPreferences, databases, and cache are wiped.
• The app behaves as if it was freshly installed.
• Login state, feature flag overrides saved locally, and cached network responses are all gone.
• managedConfig values are re-injected by the operator after the clear.

This matches the Appium capability noReset: false (which is set in the conftest.py fixtures): Appium will also reset the app at the start of each session by default.

When to use: Any test that must not be affected by data written by a previous test or previous CI run. Enable for all apps under test.

When to disable: Performance benchmarks where you need a warmed cache, or tests that explicitly verify upgrade-from-previous-version behaviour.

recycleAfterSessions¶

testConfig:
  recycleAfterSessions: 5

After the device has served n sessions its Cuttlefish pod is deleted and replaced with a fresh one. This guards against:

• Memory leaks that accumulate across sessions.
• File-system growth from logs, crash dumps, or media files that pm clear does not remove.
• Kernel or SELinux state drift caused by poorly-isolated tests.

Setting recycleAfterSessions: 1 gives maximum isolation (equivalent to a fresh VM per test run) at the cost of longer Preparing times because the Cuttlefish image must boot from scratch each time.

Recommended values:

Summary table¶