Data Logging

The Sprites add-on logs user's usage data on their local computer and they can submit their data logging file on a voluntary basis for us to debug the add-on and analyze usage behavior for research purposes. By analyzing the log file, we can get information such as the type of tables users use Sprites on, the length of each session, as well as the frequency of user errors, etc. It is important to ensure that the data logging is still working as specified if any changes are made to the add-on.

Log format

Log file

The log will be a text file log.txt containing one log json object on each new line. The most generic log json object has the following structure (the spacing and indentation will be eliminated in the actual log file since we will only store the string representation of the json object):

{
	type: <type>
	time: <datetime timestamp>
	log_id: <a sequential log id for each log items>
	sprites_id: <id for the Sprites session, None if not in Sprites mode>
	search_id: <id for the search session, None if not in search mode>
	info: <information specific to each type of object>
}

Note that for all the log objects there are a sprites_id and a search_id field. Each of these fields contains a unique identifier corresponding to a specific Sprites and search session when the actions are performed.

For sprites_id, a new one is assigned when the user toggles on the Sprites mode, and the same id will persist until the user toggles off Sprites mode.

For search_id, a new one is assigned when the user presses F to initiate a search, and it will set back to None once the user exits search mode with escape.

The log_id is for rearranging the events in proper order in case timestamps are affected by concurrency issues.

User Configuration

To keep track of the ids used across different sessions (taking into account users restarting NVDA or installing/uninstalling the add-on), we are also storing some extra information for logging in config.conf['sprites']. Below is the config spec used to define the format and type of the configuration:

confspec = {
	"userID": string
	"logID": integer
	"spritesID": integer, the last used spritesID
	"searchID": int, the last used searchID
	"firstUse": boolean, true if it is the first time users install the add-on
	"logStart": string, a date string representing the start date for the log
	"logPath": string, a path for where the log file is saved
}

Then upon starting the add-on, it will retrieve the latest ids from the configuration and use that to continue adding to the log.

Types

sprites_toggle

This includes both on and off for Sprites mode.

Fields in info:

• State: “on” or “off”

Example (shared fields like time, log_id, sprites_id, and search_id omitted here):

{
	"type": "sprites_toggle",
	"info": {"state": “on”}
}

nav

This includes regular Sprites navigation using the number keys and the five keys on the left edge of the keyboard, as well as scrolling.

Fields in info:

• Key: the name of the key being pressed (1 to 9, -, =, `, tab, capsLock, leftShift, leftControl)

• Scroll: True if a scroll key is double-pressed, False otherwise

Examples:

Single key press on key 1:

{
	"type": "nav",
	"info": {"key": “1”, "scroll": false}
}

Double press on left control to scroll:

{
	"type": "nav",
	"info": {"key": “leftControl”, "scroll": true}
}

table_info

This should be logged after Sprites mode on and captures the number of rows and columns of the table.

Fields in info:

• Row: number of rows

• Col: number of columns

Example with a table with 3 rows and 4 columns:

{
	"type": "table_info",
	"info": {"row": 3, "col": 4}
}

table_found

This should be logged when the Sprites add-on detects a table and NVDA announces “Sprites mode available” when the user is browsing with standard NVDA.

Fields in info:

• Row: number of rows

• Col: number of columns

Example with a table with 3 rows and 4 columns:

{
	"type": "table_found",
	"info": {"row": 3, "col": 4}
}

mapping

This should be logged anytime the mapping of the table has changed (Sprites mode on, after scroll or search, or jumping between different search results).

Fields in info:

• Rows: a list of mapped rows

• Cols: a list of mapped columns

Example:

{
	"type": "mapping",
	"info": {"rows": [1, 2, 3], "cols": [1, 2, 3, 4]}
}

search_toggle

This includes both “on” and “off” for Sprites search mode.

Fields in info:

• State: “on” or “off”

Example:

{
	"type": "search_toggle",
	"info": {"state": “on”}
}

search_nav

This includes jumping between search results using arrow keys.

Fields in info:

• direction: “next” or “prev”, corresponding to the next (down arrow) or previous (up arrow) search result

• Index: the search result index. For example, if the key press brings the user to the second search result, the result index will be 2.

Example:

{
	"type": "search_nav",
	"info": {"direction": “next”, "index": 2}
}

search_config

This should be logged after the user presses the search button containing the search configurations.

Fields in info:

• Case: True if case-sensitive is checked, False otherwise

• Row_f: True if row filter is applied, False otherwise

• Col_f: True if column filter is applied, False otherwise

Example with a search configuration of checking case sensitive, table row filter, and not checking table column filter:

{
	"type": "search_config",
	"info": {"case": true, "row_f": true, "col_f": false}
}

search_result

This should be logged after the search is completed, including information about the list of found locations, and the duration of search.

Fields in info:

• results: a list of tuples containing the (row, col) coordinates of cells where the search keyword is found

• duration: time spent on the search in seconds

Example with 3 search results at (1, 2), (1, 3), and (2, 4) and a search duration of 1.02 seconds.

{
	"type": "search_result",
	"info": {"results": [(1, 2), (1, 3), (2, 4)], "duration": 1.02}
}

error_exception

This should be logged if any exception occurred when Sprites mode is on.

Fields in info:

• Message: the error stack trace

Example:

{
	"type": "error_exception",
	"info": {"message": <exception stack trace string>}
}

error_edge

This should be logged if users try to press keys outside the mapped range and NVDA announces “edge of table”.

Fields in info:

• Key: the name of the key pressed

Example of pressing left control that isn’t mapped to any rows

{
	"type": "error_edge",
	"info": {"key": “leftControl”}
}

error_gesture

This should be logged if users try to press keys that are not part of the Sprites gesture and NVDA announces “gesture not available”.

Note: this won’t include keys like right shift and right control to reduce noise in data.

Fields in info:

• Key: the name of the key pressed

Example of pressing “h” which is not mapped to any available Sprites action

{
	"type": "error_gesture",
	"info": {"key": “h”}
}

error_searching

This should be logged if users try to bring up the search window while a search is happening.

Fields in info:

• No fields

Example:

{
	"type": "error_searching"
	"info": {}
}