-
Notifications
You must be signed in to change notification settings - Fork 6
Re-record sample and library baselines on an API 37 emulator #309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
aff7865
f4f7cdd
794e3ec
c1c0cc4
ad292d5
02e875d
e76a3c2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # Testify — Core Library | ||
|
|
||
| `dev.testify:testify` is the core runtime of Android Testify. It extends | ||
| [Android instrumentation tests](https://developer.android.com/training/testing/instrumented-tests) | ||
| with screenshot assertions: the library captures a bitmap of an `Activity` or `View` after all | ||
| layout and draw passes have completed, then compares that capture against a baseline image checked | ||
| into your source tree. When the rendering changes, the test fails and reports the difference. | ||
|
|
||
| Because screenshot tests are ordinary instrumented tests, they run on emulators and physical | ||
| devices from Android Studio, from the Gradle command line, and on continuous integration, and they | ||
| integrate with existing JUnit4 and Espresso test suites. | ||
|
|
||
| ## Core use cases | ||
|
|
||
| - **Assert UI rendering.** Add `ScreenshotScenarioRule` (used with `ActivityScenario`) or the | ||
| legacy `ScreenshotRule` to a test class, annotate each test method with | ||
| `@ScreenshotInstrumentation`, and call `assertSame()`. | ||
| - **Capture a subset of the screen.** Target a specific view with a view provider or root view ID | ||
| instead of capturing the full `Activity`. | ||
| - **Vary the device configuration.** Set locale, font scale, and orientation per test to verify | ||
| localization, accessibility text sizes, and rotation without maintaining separate test suites. | ||
| - **Control comparison.** Apply an exactness tolerance, exclude regions from the comparison, or | ||
| supply a custom compare method for cases where an exact pixel match is not appropriate. | ||
| - **Stabilize captures.** Suppress sources of nondeterminism — text cursors, scrollbars, password | ||
| reveal, text suggestions, and the soft keyboard — and select an alternate capture method such as | ||
| `PixelCopy` or software rendering. | ||
| - **Diagnose failures.** Enable high-contrast diff images and YAML run reports through | ||
| `TestifyFeatures` to identify which pixels changed. | ||
|
|
||
| > [!NOTE] | ||
| > This README covers the library itself. For installation, tutorials, and task-oriented guides — | ||
| > including the [**Recipes**](https://testify.dev/docs/category/recipes) section, which documents | ||
| > the scenarios listed above step by step — see [testify.dev](https://testify.dev). | ||
|
|
||
| ## Building | ||
|
|
||
| The library is built from the repository root with the Gradle wrapper. It depends on the `:Ktx` | ||
| module, so build it through the root project rather than in isolation. | ||
|
|
||
| - Build the debug variant: | ||
| ``` | ||
| ./gradlew :Library:assembleDebug | ||
| ``` | ||
|
|
||
| - Build the release AAR (written to `Library/build/outputs/aar/`): | ||
| ``` | ||
| ./gradlew :Library:assembleRelease | ||
| ``` | ||
|
|
||
| - Publish to the local Maven repository for use by a consuming project: | ||
| ``` | ||
| ./gradlew :Library:publishToMavenLocal | ||
| ``` | ||
|
|
||
| Building requires JDK 25. Warnings are treated as errors in this module, so any new compiler | ||
| warning fails the build. | ||
|
|
||
| ## Testing | ||
|
|
||
| The Testify core library is expected to pass KtLint checks, junit unit tests, and Android instrumented tests. | ||
|
|
||
| - To run the KtLint linter: | ||
| ``` | ||
| ./gradlew Library:ktlintCheck | ||
| ``` | ||
|
|
||
| - To run the junit tests: | ||
| ``` | ||
| ./gradlew Library:testDebugUnitTest | ||
| ``` | ||
|
|
||
| - To run the Android instrumented tests: | ||
| ``` | ||
| ./gradlew Library:connectedDebugAndroidTest | ||
| ``` | ||
|
|
||
| Instrumented tests require a connected device or a running emulator. They exercise the capture and | ||
| comparison pipeline against baseline images stored in `src/androidTest/assets/screenshots/`, which | ||
| are grouped by device characteristics, so results depend on the API level, resolution, density, and | ||
| locale of the target device. | ||
|
|
||
|
|
||
| # License | ||
|
|
||
| MIT License | ||
|
|
||
| Modified work copyright (c) 2022-2026 ndtp | ||
| Original work copyright (c) 2021 Shopify | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| /* | ||
| * The MIT License (MIT) | ||
| * | ||
| * Copyright (c) 2026 ndtp | ||
| * | ||
| * Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| * of this software and associated documentation files (the "Software"), to deal | ||
| * in the Software without restriction, including without limitation the rights | ||
| * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| * copies of the Software, and to permit persons to whom the Software is | ||
| * furnished to do so, subject to the following conditions: | ||
| * | ||
| * The above copyright notice and this permission notice shall be included in | ||
| * all copies or substantial portions of the Software. | ||
| * | ||
| * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | ||
| * THE SOFTWARE. | ||
| */ | ||
|
|
||
| package dev.testify.tasks.utility | ||
|
|
||
| import dev.testify.internal.Adb | ||
| import dev.testify.tasks.internal.TaskNameProvider | ||
| import dev.testify.tasks.internal.TestifyUtilityTask | ||
|
|
||
|
DanielJette marked this conversation as resolved.
|
||
| open class DisableStylusTask : TestifyUtilityTask() { | ||
|
|
||
| override fun getDescription() = "Disables stylus input on the device" | ||
|
|
||
| override fun taskAction() { | ||
| Adb(adbServiceProvider.get()).arguments( | ||
| "shell", | ||
| "settings", | ||
| "put", | ||
| "secure", | ||
| "stylus_handwriting_enabled", | ||
| "0" | ||
| ) | ||
| .execute() | ||
|
|
||
| println(" Success") | ||
| } | ||
|
|
||
| companion object : TaskNameProvider { | ||
| override fun taskName() = "disableStylusInput" | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,11 +1,12 @@ | ||||||||||||
| <resources> | ||||||||||||
| <resources xmlns:tools="http://schemas.android.com/tools"> | ||||||||||||
|
|
||||||||||||
| <!-- Base application theme. --> | ||||||||||||
| <style name="AppTheme" parent="Theme.MaterialComponents.Light.DarkActionBar"> | ||||||||||||
| <!-- Customize your theme here. --> | ||||||||||||
| <item name="colorPrimary">@color/colorPrimary</item> | ||||||||||||
| <item name="colorPrimaryDark">@color/colorPrimaryDark</item> | ||||||||||||
| <item name="colorAccent">@color/colorAccent</item> | ||||||||||||
| <item name="android:windowOptOutEdgeToEdgeEnforcement" tools:targetApi="35">true</item> | ||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the line that resolves the Compose blocker, and I think it's the right call — it fixes the whole Legacy set at once rather than just the Compose captures, and it's what keeps the new baselines comparable to the API 29 originals. Verified working: Two things I'd attach to it. It deserves a mention in the PR description. Right now the only place this appears is a commit subject. The description documents the AVD down to the VM heap but says nothing about the sample opting out of a platform behaviour — and this is the mechanism the whole baseline diff rests on. Same for the blog post, which tells readers the samples left API 29 so they'd stop "demonstrating Testify on rendering behaviour that has since moved on", while Legacy now opts out of the largest such change. One sentence — we opt Legacy out deliberately, to keep the baseline diff reviewable — reconciles the two and saves the next person a bisect. It's an escape hatch with a shelf life.
Suggested change
Related, and the reason I'd not let this sit forever: |
||||||||||||
| </style> | ||||||||||||
|
|
||||||||||||
| <style name="AppTheme.NoActionBar"> | ||||||||||||
|
|
||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pinning to
@2.0, moving togoogle_apis_ps16kand adding the settle step all look right — and I checked the inputs againstndtp/android-avd-manager-action@2.0'saction.yml:api-level,target,channel,arch,profile,ram-sizeanddisk-sizeare all valid, and the2.0tag exists.37.2-beta3still reportsSDK_INT == 37, soDeviceStringFormatter.androidVersionkeeps producing the37-…key and the baselines still resolve. No concern there.The question is the divergence this creates. The comment states plainly that:
Meanwhile API 37.0 is what everything else in the repo prescribes —
bitrise.yml:124(api_level: '37.0'),docs/docs/get-started/2-configuring-an-emulator.md,RECIPES.md:10, and the blog post's setup instructions.So either:
swiftshader_indirect), in which case 37.0 is fine locally and on Bitrise and it's worth saying so in the comment — otherwise the next person reads this and assumes the documented local setup is broken; or_emulatorSetupis running on a known-crashy image and the contributor docs are sending people to one.Worth resolving either way — right now the repo simultaneously documents 37.0 as the target and 37.0 as broken.
Minor:
enable-hw-keyboarddefaults tofalsein this action andcoresdefaults to2, while the emulator docs specify keyboard input enabled and 4 cores. Pre-existing and almost certainly irrelevant for Flix (no text input), so only worth touching if you're already in here.