Repository navigation
fix: support StarNodes INT8 ConvRot safetensors metadata #2178
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -247,6 +247,50 @@ bool read_safetensors_file(const std::string& file_path, | |
| comfy_quant_configs.emplace(module_name, std::move(config)); | ||
| } | ||
|
|
||
| // StarNodes stores INT8 ConvRot configuration in safetensors file metadata | ||
| // rather than in per-layer .comfy_quant tensors. | ||
| auto metadata_item = header_.find("__metadata__"); | ||
| if (metadata_item != header_.end() && metadata_item->is_object()) { | ||
| auto quantization_item = metadata_item->find("_quantization_metadata"); | ||
| if (quantization_item != metadata_item->end() && quantization_item->is_string()) { | ||
| try { | ||
| const nlohmann::json quantization = nlohmann::json::parse(quantization_item->get<std::string>()); | ||
| auto layers = quantization.find("layers"); | ||
| if (layers != quantization.end() && layers->is_object()) { | ||
| const std::string format = quantization.value("format", ""); | ||
| for (const auto& layer : layers->items()) { | ||
| const std::string& module_name = layer.key(); | ||
| if (!layer.value().is_object() || | ||
| header_.find(module_name + ".weight") == header_.end() || | ||
| header_.find(module_name + ".weight_scale") == header_.end() || | ||
|
Comment on lines
+264
to
+265
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: set -eu
printf '%s\n' '--- diff ---'
git diff --no-ext-diff --unified=80 0518b34374761cb82aeda5ad3c2f0686fdad2cec b00672d4a273a0cf275b0311181c5f2959600c57 -- src/model_io/safetensors_io.cpp
printf '%s\n' '--- current surrounding source ---'
nl -ba src/model_io/safetensors_io.cpp | sed -n '210,315p'
printf '%s\n' '--- repository references ---'
rg -n -F --glob '!build/**' --glob '!dist/**' -e '_quantization_metadata' -e 'int8_convrot' -e 'model.diffusion_model' -e 'comfy_quant' .Repository: leejet/stable-diffusion.cpp Length of output: 38016 🌐 Web query:
💡 Result: 🏁 Script executed: set -eu
python3 - <<'PY'
import urllib.request
urls = [
"https://raw.githubusercontent.com/Starnodes2024/comfyui-starnodes-modelconverter/main/README.md",
"https://api.github.com/repos/Starnodes2024/comfyui-starnodes-modelconverter/contents",
]
for url in urls:
print("URL:", url)
try:
with urllib.request.urlopen(url, timeout=10) as r:
data = r.read().decode("utf-8", "replace")
print(data[:20000])
except Exception as e:
print("ERROR:", type(e).__name__, str(e))
PYRepository: leejet/stable-diffusion.cpp Length of output: 35965 🏁 Script executed: set -eu
nl -ba src/model_io/safetensors_io.cpp | sed -n '350,420p'
nl -ba src/model_io/safetensors_io.cpp | sed -n '100,170p'Repository: leejet/stable-diffusion.cpp Length of output: 7113 Match StarNodes metadata names to their tensor names. StarNodes AIO metadata strips Normalize only the documented prefixes before matching. Accept a normalized name only when its 🤖 Prompt for AI Agents |
||
| comfy_quant_configs.find(module_name) != comfy_quant_configs.end()) { | ||
| continue; | ||
| } | ||
|
|
||
| ComfyQuantConfig config; | ||
| config.format = layer.value().value("format", format); | ||
| config.convrot = config.format == "int8_convrot" || layer.value().value("convrot", false); | ||
| config.group_size = layer.value().value("convrot_groupsize", | ||
| layer.value().value("group_size", 0)); | ||
|
Comment on lines
+273
to
+274
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Use the ConvRot default when the group size is absent. If an 🤖 Prompt for AI Agents |
||
| if (config.format == "int8_convrot") { | ||
| config.format = "int8_tensorwise"; | ||
| } | ||
| if (config.format == "int8_tensorwise") { | ||
| #ifdef SD_USE_UPSTREAM_GGML | ||
| set_error(error, "INT8 tensorwise/convrot is not supported by this ggml build (tensor '" + module_name + ".weight')"); | ||
| return false; | ||
| #endif | ||
| comfy_quant_configs.emplace(module_name, std::move(config)); | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| } | ||
| } | ||
| } | ||
| } catch (const std::exception&) { | ||
| set_error(error, "parsing StarNodes quantization metadata failed: '" + file_path + "'"); | ||
| return false; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| tensor_storages.clear(); | ||
| for (auto& item : header_.items()) { | ||
| std::string name = item.key(); | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Do not reject valid per-layer configurations because optional metadata is malformed.
If a checkpoint has valid
.comfy_quanttensors and a malformed_quantization_metadatastring, this new parse fails before the reader can use the preferred per-layer configurations. The same checkpoint previously reached tensor loading. Ignore an unusable optional layer map when the applicable per-layer configurations are already available, or limit the error to a layer that requires the map. ComfyUI documents.comfy_quantas the per-module configuration. (github.com)Also applies to: 283-285
🤖 Prompt for AI Agents