Skip to content

Add configurable table header reading and row/column coordinates - #103

Open
amanKG777 wants to merge 2 commits into
trypsynth:masterfrom
amanKG777:table-headers
Open

amanKG777 wants to merge 2 commits into
trypsynth:masterfrom
amanKG777:table-headers

Conversation

@amanKG777

Copy link
Copy Markdown
Contributor

Summary

Adds configurable table header reading (announcing column headers before or after cell data, or off) and optional row/column coordinate announcements under Backtalk Settings > Verbosity. When navigating tables (including web tables in Chrome/WebViews and native grids), cell data is contextualized with its corresponding column header separated by a natural speech pause, making tabular data (such as scorecards, timetables, and financial reports) much easier to follow.

Technical Implementation & Architecture

  1. Verbosity Preferences (verbosity_preferences.xml, TalkBackService.java):

    • Added table_column_headers list preference under Verbosity with options: after (default), before, and off.
    • Added table_speak_row_column_numbers switch to toggle coordinate announcements (Row X, Column Y).
    • Wired preferences into TalkBackService.reloadPreferences() and propagated them to GlobalVariables. Added preferences and strings to both phone and wear XML resources.
  2. Table Role Detection & Classification (AccessibilityNodeInfoUtils.java, CollectionState.java):

    • Made isTableRoot and isTableCellUnderTable public in AccessibilityNodeInfoUtils.
    • Enhanced isTableRoot to accept containers with Role.ROLE_GRID as well as any CollectionInfo having row/column counts > 1 (including dynamic tables where row/column count is -1), ensuring web tables inside WebViews/Chrome are consistently detected as tables.
    • Updated CollectionState.getCollectionRole() and transitions (NAVIGATE_ENTER, NAVIGATE_INTERIOR) to route isTableRoot containers into the table state machine.
    • Extended CollectionState.getTableItemState() with fallback to getTableCellUnderTable() when focus lands on inner text elements within a cell.
    • Updated getTableHeadingType so row 0 in multi-row tables is recognized as column headers (TYPE_COLUMN), even when the host application or browser does not explicitly set isHeading.
  3. Header Resolution & Cache (CollectionState.java, CollectionStateFeedbackUtils.java):

    • Exposed getColumnHeaders() and getRowHeaders() on CollectionState.
    • Added findCellAt(tableRoot, targetRow, targetCol) in CollectionStateFeedbackUtils to query the accessibility tree for the corresponding cell at (0, columnIndex) if the cached header is not yet populated, dynamically caching column titles.
  4. Speech Output Composition & Pauses (CollectionStateFeedbackUtils.java):

    • Refactored getTableItemCellFeedback() to format speech according to verbosity preferences:
      • Headers After: [coordinates] [cell data]. [column header] (e.g. Column 2, 162. Runs scored.).
      • Headers Before: [coordinates] [column header]. [cell data] (e.g. Runs scored, Column 2. 162.).
      • Headers Off: [coordinates] [cell data] or cell data alone.
    • Punctuation (. ) is enforced between cell text and headers to ensure the TTS engine inserts a natural speech pause rather than running them together.
    • Row 0 cells announce their coordinate, text, and role (Column heading when roles are enabled).
  5. Row Boundary & Focus Tracking (GlobalVariables.java, Feedback Rules):

    • Added row index tracking in GlobalVariables to detect row changes (isRowTransition), speaking the row coordinate when moving between rows and omitting redundant row coordinates when swiping across columns in the same row.
    • Updated EventTypeViewAccessibilityFocusedFeedbackRule and EventTypeViewAccessibilityFocusedFeedbackRuleForTV to check globalVariables.isFocusedNodeInTable(node), routing table cell feedback through getTableItemCellFeedback and suppressing redundant generic collection transitions.

@aaron-gh

aaron-gh commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

I reviewed the code and tested it on an Android 16 emulator, in Chrome and in a native 3-column RecyclerView grid, against its base commit (d3bbe47), with verbose logging. In the examples, "before" is the base and "after" is this PR.

Behaviour

Tables without a header row

getTableHeadingType now makes every row-0 cell a column header, and getTableItemCellFeedback treats row 0 of any table with more than one row as headings. For a plain 3×3 table (Apple / Red / 3, Banana / Yellow / 5, …):

  • Before: "Apple", "Row 1, Column 1, In table, 3 rows, 3 columns", "Red", "Column 2", …
  • After: "Row 1, Column 1, " (Apple is never spoken), "Column 2, ", then "Row 2, Column 1, Banana. Apple", "Column 2, Yellow. Red", "Column 3, 5. 3". "In table, 3 rows, 3 columns" is no longer said.

The same happens in native grids. In a photo-style grid, the first row's items become "Row 1, Column 2, Photo 2, Column heading", and every other item gets the first-row item in its column as a header: "Row 4, Column 2, Photo 11. Photo 2". Before: "Photo 10, Row 4, Column 1", "Photo 11, Column 2".

Header cells lose their text

Cells in a real header row (<th>) are spoken as "Row 1, Column 2, " with the header name missing. In a table with Name / Count / Score / Note headers, none of the four names is spoken.

Spanning headers

With a header row of "First half" and "Second half" (each spanning two columns) above Q1–Q4, the Q1–Q4 headers are never used for data cells. Columns 2 and 4 get "First half" and "Second half", columns 3 and 5 get no header, and "Q1, column header" is itself followed by "First half". Dropping the getRowSpan() == 1 && getColumnSpan() == 1 check in getTableHeadingType maps a spanning header to its first column only.

The row and column numbers switch has no effect

The switch is read with VerbosityPreferences.getPreferenceValueBool. Once a preset is stored, and opening the Verbosity screen stores the default one, that reads pref_verbosity_preset_value_custom_pref_table_speak_row_column_numbers_key. The screen only writes keys under that name for the entries in VerbosityPrefFragment.switchPreferenceKeyValueMap, and this one isn't there, so the switch writes the plain key. On the emulator, the switch shows off and every cell still says "Row N, Column N". On the High and Low presets it's forced on or off regardless.

Other settings

  • "Speak container info" off no longer removes row and column information in tables. The table branch replaces getCollectionItemTransitionDescription, which was the part that setting controlled, and doesn't check it.
  • ensureTerminalPunctuation adds a period to make a pause. With punctuation set to All, every punctuation mark in an announcement is read by name, so this becomes "162 period Runs".

Defaults

With the defaults (headers after, numbers on), every table and grid sounds different from today for everyone. Row header names, such as "India", are only spoken when numbers are off, so by default they're replaced by "Row N". Column numbers and headers are spoken on every cell, not only when the column changes.

Code

Composition changes state

GlobalVariables.getTableItemCellFeedback updates lastTableItemRowIndex, lastTableItemColIndex and lastTableRoot while composing an announcement. Backtalk composes announcements for the next and previous swipe targets in advance (#43, EventFilter.prepareFocusSpeech), and restores the focus state afterwards with SavedFocusState, which doesn't include these fields. So the last prepared target decides the "previous row". In the native grid, "Row 3, Column 2" is followed by "Row 3, Column 3", and going back the row number appears and disappears. Web tables are only unaffected because nothing is prepared for web content. The row change should come from CollectionState, which is already saved and restored, rather than from state written during composition.

Text loses its spans

getTableItemCellFeedback turns the cell text into a String (cellContent.toString().trim(), and ensureTerminalPunctuation(...) + " " + ...), so the result loses every span. That includes the LocaleSpans from the node's text, which AccessibilityNodeFeedbackUtils adds for the language switcher and apps can set themselves, and which LanguageSwitch, FailoverTextToSpeech and EmojiSpeech use to pick the language. Table cells would then always be spoken in the default language. Keep the CharSequences and join them with CompositorUtils.joinCharSequences.

A cell from another table

CollectionState.getTableItemState now falls back to getTableCellUnderTable(announcedNode) when no collection item under collectionRoot is found. That cell can belong to a different table, such as a table inside a list item. Its indexes are then used with this collection's CollectionInfo, and its row and column names are written into this collection's header caches.

Header text

getHeaderText used to stop at any node with more than one child, so it wouldn't pick an unrelated label. It now returns the first child with text, which for a header cell with an icon before its label can be the icon's description. It's also used for the existing header caching, not only the new code.

What counts as a table

isTableRoot now also accepts collections with a row or column count of -1. getCollectionRole() reports those, and any pager with both counts above 1, as ROLE_GRID. Role deliberately keeps staggered grids and unknown-count collections out of ROLE_GRID, because they "don't always map neatly to row and column semantics". These methods are also used for:

  • table navigation, and whether the Rows and Columns reading controls are offered
  • the braille display: NodeBrailler labels grids from getCollectionRole(), and skips a cell's column label when its heading type is TYPE_COLUMN, which now includes every row-0 cell

So this changes navigation and braille output as well as speech.

Smaller things

  • lastTableRoot keeps a node for the service's lifetime, and nothing clears it when the window changes.
  • Unused: lastTableItemColIndex (written, never read), the no-argument isFocusedNodeInTable(), GlobalVariables.getCollectionState() and CollectionState.getRowHeaders(). isTableCell is made public but is only used inside AccessibilityNodeInfoUtils.
  • findCellAt has the Javadoc of getTableItemCellFeedback.
  • No unit tests for the formatting or the header detection, and no differences.md entry.

The options themselves are useful: headers before, after or off, and a numbers switch. I'd suggest building them on top of the existing getCollectionItemTransitionDescription output, leaving header detection, isTableRoot and getCollectionRole as they are. Use the existing defaults, so nothing changes until someone picks an option. Add the switch to switchPreferenceKeyValueMap, and keep composition free of state changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants