|
2 | 2 |
|
3 | 3 | ## deglib v0.2.1 |
4 | 4 |
|
5 | | -### Highlights |
6 | | -* **New Distance Metrics:** |
7 | | - * **`FP16_L2` Metric:** Added native IEEE 754 half-precision Euclidean distance (`Metric::FP16_L2`) with AVX-512, AVX2 (F16C/FMA), and Scalar kernels, fully integrated into C++ `FloatSpace` and Python bindings (`deglib.Metric.FP16_L2`). |
8 | | - * **`Int8_InnerProduct` Metric:** Added signed 8-bit integer inner product (`Metric::Int8_InnerProduct`) with AVX-512, AVX512-VNNI, AVX2, AVX2-VNNI, and Scalar kernels. |
9 | | - * **`Int8_L2` Metric:** Added signed 8-bit integer L2 distance (`Metric::Int8_L2`) with AVX-512, AVX2, and Scalar kernels, including Python bindings support (`np.int8`). |
10 | | - * **`Uint8_InnerProduct` Metric:** Added native 8-bit unsigned integer inner product distance (`Metric::Uint8_InnerProduct`) with AVX-512, AVX512-VNNI, AVX2, AVX2-VNNI, and Scalar kernels. |
11 | | -* **Scalar Quantization (`deglib::optimization::quantization`):** |
12 | | - * Added `SymCalibratorInt8` for symmetric Int8 quantization ($[-127, 127]$). |
13 | | - * Added `AffineCalibratorUint8` for affine/asymmetric UInt8 quantization ($[0, 255]$ with scale and offset). |
14 | | - * Python APIs exposed via `deglib.optimization.quantize_sym_int8(data)` and `deglib.optimization.quantize_affine_uint8(data)`. |
15 | | -* **SIMD & VNNI Optimizations:** |
16 | | - * Added dedicated AVX-VNNI and AVX512-VNNI hardware acceleration for 8-bit integer dot products (`Int8_IP` and `UInt8_IP`). |
17 | | - * Accelerated 8-bit integer distance computations with optimized SIMD horizontal reductions in AVX-512 (`_mm512_reduce_add_epi32`) and AVX2. |
18 | | - * Centralized CPU instruction set detection and fallback resolution in `deglib::cpu::resolve_instruction_set()`. |
19 | | - * Fixed SIMD instruction fallback selection in distance kernels when running on VNNI-capable CPUs. |
20 | | -* **Cross-Platform & Header Consistency Fixes:** |
21 | | - * Added missing `<stdexcept>` header in `cpu.h` fixing `std::runtime_error` compilation errors on GCC and Clang (Linux / macOS). |
22 | | - * Added missing `<cstring>` includes across integer distance headers for portable builds. |
23 | | - * Fixed GCC/Clang inlining error (`target specific option mismatch` in `_mm256_cvtph_ps`) by including `avx2,f16c,fma` in `DEGLIB_TARGET_AVX512` and `DEGLIB_TARGET_AVX512_VNNI` target attributes. |
24 | | - * Renamed `evp_inner_product.h` to `evp_ip.h` for consistent header naming convention across distance modules. |
25 | | -* **Python Memory Leak Fix:** Resolved memory leak in Python bindings when sequentially loading multiple graph instances from disk. |
26 | | -* **VIBE Benchmark Suite (`examples/vibe`):** |
27 | | - * Added complete Vector Index Benchmark (VIBE) runner with automated Hugging Face HDF5 dataset downloads, grid evaluations, Pareto frontier plotting, and default FLAS pre-sorting. |
28 | | - * Extended VIBE DegANN with support for query feature quantization, FP16 reranking, and MRNG edge pruning. |
29 | | - * Enhanced benchmark visualizer with color-coded targets, $k$-intensity curves, and dashed styling for pruned graphs. |
30 | | -* **Sliding Window Streaming Demo (`examples/sliding_window`):** Added an interactive dynamic graph benchmark showcasing real-time sliding window ingestion (continuous additions and deletions). |
31 | | -* **Build & CI Improvements:** |
32 | | - * Configured MSVC `/MP` (multi-threaded compilation) and `/bigobj` (extended object section limit) in CMake compilation options. |
33 | | - * Updated Read the Docs webhook trigger in CI to pass authentication tokens as POST form data. |
34 | | - * Updated `python/README.md` and removed obsolete memory safety documentation. |
| 5 | +### Overview |
| 6 | +deglib v0.2.1 expands metric support with native half-precision and 8-bit integer formats, introduces scalar quantization tools with AVX-VNNI hardware acceleration, fixes a sequential memory leak in Python bindings, and adds the comprehensive VIBE benchmark suite. |
| 7 | + |
| 8 | +--- |
| 9 | + |
| 10 | +### 🚀 Key Features & Improvements |
| 11 | + |
| 12 | +#### New Distance Metrics |
| 13 | +* **`FP16_L2` Metric:** Native IEEE 754 half-precision Euclidean distance (`Metric::FP16_L2`) with AVX-512, AVX2 (F16C/FMA), and Scalar kernels, fully integrated into C++ `FloatSpace` and Python bindings (`deglib.Metric.FP16_L2`). |
| 14 | +* **`Int8_InnerProduct` Metric:** Signed 8-bit integer inner product (`Metric::Int8_InnerProduct`) with AVX-512, AVX512-VNNI, AVX2, AVX2-VNNI, and Scalar kernels. |
| 15 | +* **`Int8_L2` Metric:** Signed 8-bit integer Euclidean distance (`Metric::Int8_L2`) with AVX-512, AVX2, and Scalar kernels, including Python bindings support (`np.int8`). |
| 16 | +* **`Uint8_InnerProduct` Metric:** 8-bit unsigned integer inner product distance (`Metric::Uint8_InnerProduct`) with AVX-512, AVX512-VNNI, AVX2, AVX2-VNNI, and Scalar kernels. |
| 17 | + |
| 18 | +#### Scalar Quantization (`deglib::optimization::quantization`) |
| 19 | +* **Symmetric Int8 Quantization:** Added `SymCalibratorInt8` for signed Int8 mapping ($[-127, 127]$), ideal for Cosine and Inner Product spaces. |
| 20 | +* **Affine UInt8 Quantization:** Added `AffineCalibratorUint8` for unsigned 8-bit mapping ($[0, 255]$ with dynamic scale and offset calibration), optimized for Euclidean ($L_2$) search. |
| 21 | +* **Python APIs:** Exposed via `deglib.optimization.quantize_int8()` and `deglib.optimization.quantize_uint8()`. |
| 22 | + |
| 23 | +#### SIMD & Hardware Acceleration (AVX-VNNI) |
| 24 | +* **VNNI Acceleration:** Dedicated AVX-VNNI and AVX512-VNNI hardware vector dot product instructions for 8-bit integer inner products (`Int8_IP` and `UInt8_IP`). |
| 25 | +* **Fast Horizontal Reductions:** Accelerated integer distance evaluation using optimized SIMD horizontal reductions in AVX-512 (`_mm512_reduce_add_epi32`) and AVX2. |
| 26 | +* **Centralized CPU Dispatch:** Streamlined CPU instruction set detection and runtime fallback resolution in `deglib::cpu::resolve_instruction_set()`. |
| 27 | + |
| 28 | +#### Benchmarks & Example Workflows |
| 29 | +* **VIBE Benchmark Suite (`examples/vibe`):** Added Vector Index Benchmark runner with automated Hugging Face dataset downloads, grid evaluations, interactive Plotly HTML visualizations with GUI log exploration, query feature quantization, and FP16/FP32 candidate reranking. |
| 30 | +* **Sliding Window Streaming Benchmark (`examples/sliding_window`):** Added dynamic streaming experiment reproducing continuous updates against DEG from the CleANN benchmark. |
| 31 | + |
| 32 | +--- |
| 33 | + |
| 34 | +### 🛠️ Bug Fixes & Stability |
| 35 | + |
| 36 | +* **Python Memory Leak:** Resolved a memory leak in Python bindings when sequentially loading or converting multiple graph instances from disk. |
| 37 | +* **Compiler Compatibility (GCC / Clang):** |
| 38 | + * Added missing `<stdexcept>` header in `cpu.h` fixing `std::runtime_error` compilation failures on Linux and macOS. |
| 39 | + * Added missing `<cstring>` headers across integer distance implementations for standard portability. |
| 40 | + * Fixed GCC/Clang inlining errors (`target specific option mismatch` in `_mm256_cvtph_ps`) by including `avx2,f16c,fma` in `DEGLIB_TARGET_AVX512` target attributes. |
| 41 | +* **Header Consistency:** Renamed `evp_inner_product.h` to `evp_ip.h` for uniform naming across all distance headers. |
| 42 | +* **Deterministic CI Tests:** Configured fixed seeds across graph integration tests to guarantee 100% test reproducibility across all platforms and Python versions. |
| 43 | + |
| 44 | +--- |
| 45 | + |
| 46 | +### 🐍 Python API Summary |
| 47 | + |
| 48 | +* **Quantization Functions:** |
| 49 | + * `deglib.optimization.quantize_int8(vectors, drop_ratio=0.0, num_threads=0) -> np.ndarray` |
| 50 | + * `deglib.optimization.quantize_uint8(vectors, per_dim=False, drop_ratio=0.0, num_threads=0) -> np.ndarray` |
| 51 | +* **New Metrics:** `Metric.FP16_L2`, `Metric.Int8_InnerProduct`, `Metric.Int8_L2`, `Metric.Uint8_InnerProduct`. |
| 52 | +* **Reranking:** `deglib.search.rerank(space, queries, candidate_indices, base_vectors, k_top=..., return_distances=...)` |
| 53 | + |
35 | 54 | --- |
36 | 55 |
|
37 | 56 | ## deglib v0.2.0 |
|
0 commit comments