Skip non-numeric covariates when computing standardized differences - #1044
Open
arpitjain099 wants to merge 1 commit into
Open
Skip non-numeric covariates when computing standardized differences#1044arpitjain099 wants to merge 1 commit into
arpitjain099 wants to merge 1 commit into
Conversation
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_get_numeric_varsis documented as working out which covariates are numeric, but it never looks at the dtype. A string column with at leastthresholddistinct values passes the continuous test, and the standardized difference then dies inside numpy:That reaches users through
plot_ps_diagnostics, which takes whatever covariate columns the caller names, and categorical covariates are a normal thing to have in a propensity model. The function already has the right behaviour for columns it cannot use: it drops them and logs which ones. Non-numeric columns now take that path, so the diagnostic reports balance for the covariates it can measure instead of failing outright.The dropped-variables message was only describing two of its three reasons, so it now mentions the non-numeric case too.
Two tests: a frame with a string covariate classifies as
['age']continuous and['flag']binary and produces finite differences, and a frame whose only covariate is non-numeric still raises the existingValueErrorfor having nothing to measure. The first fails on master withassert ['age', 'group'] == ['age']and then theTypeErrorabove.pytest tests/test_visualize.py tests/test_visualize_score_ci.pyis 33 passing, and black is clean.