Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
c1ed2ff
fix: drop unwired policy fields and encoding/embedding NaN bugs
mkumar73 Sep 4, 2026
150a012
fix: use configured dtype in output-budget memory estimate
mkumar73 Sep 4, 2026
769e7b1
feat: default Preprocessor output to a single array
mkumar73 Sep 5, 2026
f53673d
ci: CI before publish and test minimum dependencies
mkumar73 Sep 5, 2026
4641ab6
fix: raise scikit-learn minimum and use scipy trapezoid
mkumar73 Sep 5, 2026
9cc5316
chore: formatting
mkumar73 Sep 5, 2026
b6fc3e7
docs: fix broken source-install command in README
mkumar73 Sep 5, 2026
8a948c5
fix: remove unimplemented resolution stubs from public placement API
mkumar73 Sep 5, 2026
ec5e3ce
chore: add py.typed marker for PEP 561 compliance
mkumar73 Sep 5, 2026
4fe150d
docs: remove custombin from categorical_method docstring
mkumar73 Sep 5, 2026
824102e
fix: validate feature_preprocessing keys against input columns
mkumar73 Sep 5, 2026
0c4ccc7
fix: numerical_method=none no longer applies scaling
mkumar73 Sep 5, 2026
1139b92
fix: add missing polars optional dependency and its tests
mkumar73 Sep 5, 2026
3f42af9
chore: test case correction
mkumar73 Sep 6, 2026
5e81a3b
docs: note Nystroem's non-uniform pointwise approximation error
mkumar73 Sep 6, 2026
dc0386e
fix: document periodic transform wrap-around and correct binning docs…
mkumar73 Sep 6, 2026
2fe4e0d
test: add shared penalty matrix symmetry, PSD, rank, and rescaling ch…
mkumar73 Sep 6, 2026
948bb14
docs: remove stale claim from reproducibility note
mkumar73 Sep 6, 2026
0508e56
fix: clip P-spline and tensor-product out-of-range transforms via policy
mkumar73 Sep 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ name: CI

on:
workflow_dispatch:
workflow_call:
push:
branches:
- main
Expand Down Expand Up @@ -150,6 +151,38 @@ jobs:
- name: Run unit tests
run: poetry run pytest tests/ -v

min-deps:
name: Minimum supported dependencies
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install Poetry
run: pipx install poetry

- name: Configure Poetry
run: poetry config virtualenvs.in-project true

- name: Install dependencies
run: poetry install

- name: Force the declared minimum runtime dependency versions
run: |
poetry run pip install \
"numpy==1.24.*" \
"pandas==2.0.*" \
"scipy==1.10.*" \
"scikit-learn==1.6.*"

- name: Run unit tests against the minimum versions
run: poetry run pytest tests/ -q

smoke:
name: Smoke tests (Python 3.12, ubuntu)
runs-on: ubuntu-latest
Expand Down Expand Up @@ -248,6 +281,8 @@ jobs:
module: sentence_transformers
- extra: lightgbm
module: lightgbm
- extra: polars
module: polars

steps:
- uses: actions/checkout@v4
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/publish-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,14 @@ permissions:
id-token: write

jobs:
ci:
name: Full CI (required before publish)
uses: ./.github/workflows/ci.yml
secrets: inherit

publish:
runs-on: ubuntu-latest
needs: ci
environment: pypi-publish
# The "v*.*.*" trigger also matches RC tags (e.g. v2.0.0rc2), so guard
# against publishing pre-releases to real PyPI. RC tags are handled by
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/publish-testpypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,14 @@ permissions:
id-token: write

jobs:
ci:
name: Full CI (required before publish)
uses: ./.github/workflows/ci.yml
secrets: inherit

publish-rc:
runs-on: ubuntu-latest
needs: ci
environment: testpypi-publish

steps:
Expand Down
18 changes: 10 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,10 @@ y = np.random.randn(100)
# Global strategies: PLE for numerics, integer codes for categoricals
preprocessor = Preprocessor(numerical_method="ple", categorical_method="int")

X = preprocessor.fit_transform(df, y) # dict of transformed feature blocks
X = preprocessor.fit_transform(df, y) # single stacked array, one row per sample

print({k: v.shape for k, v in X.items()})
# {'num_age': (100, 7), 'num_income': (100, 7), 'cat_city': (100, 1)}
print(X.shape)
# (100, 15)
```

> **Note:** PreTab accepts a `pandas.DataFrame` or a `numpy.ndarray` and infers numerical
Expand Down Expand Up @@ -185,7 +185,7 @@ pip install "pretab[all]" # both of the above
```bash
git clone https://github.com/OpenTabular/PreTab
cd PreTab
pip install -e ".[dev]"
poetry install
```

## Usage
Expand All @@ -209,8 +209,8 @@ preprocessor = Preprocessor(
task="regression",
)

X_dict = preprocessor.fit_transform(df, y) # {"num_age": ..., "cat_city": ...}
X_array = preprocessor.transform(df, return_array=True) # single stacked ndarray
X_array = preprocessor.fit_transform(df, y) # single stacked ndarray
X_dict = preprocessor.transform(df, return_array=False) # {"num_age": ..., "cat_city": ...}

preprocessor.get_feature_info(verbose=True) # inspect resolved strategies
```
Expand All @@ -227,8 +227,10 @@ experience numerical imputer -> minmax -> quantile 1 -
city categorical imputer -> onehot -> to_float 4 4
```

> **Note:** `transform` returns a dict of feature blocks by default (keys prefixed `num_`
> and `cat_`), or a single stacked array when you pass `return_array=True`.
> **Note:** `transform` returns a single stacked array by default, so a `Preprocessor` drops
> straight into a plain `sklearn.pipeline.Pipeline`. Pass `output_structure="blocks"` (or
> `return_array=False` for a single call) for the dict-of-feature-blocks form instead (keys
> prefixed `num_` and `cat_`).

### Standalone transformers

Expand Down
1 change: 1 addition & 0 deletions docs/core_concepts/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ in depth.
| `target_aware`, `placement_strategy` | `True`, `"cart"` | [Target awareness](target_awareness.md) |
| `numerical_imputation`, `categorical_imputation`, `add_missing_indicator` | `"median"`, `"most_frequent"`, `False` | [Missing values](missing_values.md) |
| `output_format`, `dtype` | `"dense"`, `None` | [Outputs and inspection](outputs_and_inspection.md) |
| `output_structure` | `"matrix"` | [Outputs and inspection](outputs_and_inspection.md) |
| `random_state` | `None` | [Reproducibility](reproducibility.md) |

## Where to go next
Expand Down
Loading
Loading