Skip to content

Skip non-numeric covariates when computing standardized differences - #1044

Open
arpitjain099 wants to merge 1 commit into
uber:masterfrom
arpitjain099:fix/std-diffs-non-numeric
Open

Skip non-numeric covariates when computing standardized differences#1044
arpitjain099 wants to merge 1 commit into
uber:masterfrom
arpitjain099:fix/std-diffs-non-numeric

Conversation

@arpitjain099

Copy link
Copy Markdown
Contributor

_get_numeric_vars is documented as working out which covariates are numeric, but it never looks at the dtype. A string column with at least threshold distinct values passes the continuous test, and the standardized difference then dies inside numpy:

X = pd.DataFrame({'age': ..., 'group': rng.choice(list('abcdef'), size=n), 'flag': ...})
_get_numeric_vars(X)     # (['age', 'group'], ['flag'])
get_std_diffs(X, w)      # TypeError: ufunc 'divide' not supported for the input types

That reaches users through plot_ps_diagnostics, which takes whatever covariate columns the caller names, and categorical covariates are a normal thing to have in a propensity model. The function already has the right behaviour for columns it cannot use: it drops them and logs which ones. Non-numeric columns now take that path, so the diagnostic reports balance for the covariates it can measure instead of failing outright.

The dropped-variables message was only describing two of its three reasons, so it now mentions the non-numeric case too.

Two tests: a frame with a string covariate classifies as ['age'] continuous and ['flag'] binary and produces finite differences, and a frame whose only covariate is non-numeric still raises the existing ValueError for having nothing to measure. The first fails on master with assert ['age', 'group'] == ['age'] and then the TypeError above.

pytest tests/test_visualize.py tests/test_visualize_score_ci.py is 33 passing, and black is clean.

Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant