Skip to content

Add Int4BlockCompressor utility for quantized tensor serialization - #359

Open
AhmadHassan-BTed wants to merge 2 commits into
SMILELab-FL:masterfrom
AhmadHassan-BTed:feature/int4-block-compressor
Open

AhmadHassan-BTed wants to merge 2 commits into
SMILELab-FL:masterfrom
AhmadHassan-BTed:feature/int4-block-compressor

Conversation

@AhmadHassan-BTed

Copy link
Copy Markdown

Added Int4BlockCompressor in fedlab/utils/compressor.py to compress serialized model parameters and gradients down to packed 4-bit integers with per-block scaling.

Background:
SerializationTool converts model parameters and gradients into flat 1D float32 tensors, and ModelMaintainer.shape_list notes intended use for tensor compression. Transmitting uncompressed 32-bit floats over network rounds is bandwidth heavy in distributed federated settings.

Changes:

  • Added Int4BlockCompressor in fedlab/utils/compressor.py with:
    • Block-level dynamic scaling (scale = max(|x|) / 7.0).
    • INT4 quantization with two 4-bit values packed into one uint8 byte (~8x payload size reduction).
    • Optional residual error feedback tracking to mitigate gradient drift across communication rounds.
    • Convenience methods compress_tensor, decompress_tensor, compress_model, deserialize_model, compress_model_gradients, and deserialize_model_gradients.
  • Exported Int4BlockCompressor in fedlab/utils/init.py.
  • Added unit tests in tests/test_utils/test_compressor.py covering shapes, padding, reconstruction MSE, error feedback accumulation, and model serialization pipelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant