feat(lancedb): enable GooseFS config in UI and fill missing DB icons

[XuQianJin-Stars]

Summary

Shares GooseFS/COS storage_options resolution so the UI and CLI both pick up GOOSEFS_* env vars, exposes IVF_HNSW_SQ / IVF_HNSW_PQ in Run Test, enables concurrent loads for LanceDB, and reuses existing icons for DBs without dedicated assets.

What's added

• Shared storage options: move GooseFS/COS resolution into vectordb_bench/backend/clients/lancedb/config.py as build_lancedb_storage_options() / build_goosefs_storage_options() / build_cos_storage_options(); CLI (cli.py) now reuses the same path.
• UI GooseFS support: LanceDBConfig.to_dict() resolves storage_options from URI + env; hide storage_options via ui_hidden_configs(); skip empty optional fields (token, storage_options) with _extra_empty_skip.
• Run Test index types: expose IVF_HNSW_SQ / IVF_HNSW_PQ in dbCaseConfigs.py (replace the old HNSW alias) with corrected param visibility for IVF / PQ / HNSW groups.
• Concurrent load: set LanceDB.thread_safe = True so ConcurrentInsertRunner can use max_workers > 1.
• DB icons: map missing icons in styles.py (TencentElasticsearch, AliSQL, VectorChord, Pinot, SeekDB, VolcMySQL, Doris) to existing assets.

Key improvements

• UI/CLI parity: goosefs:// URIs in Run Test now get the same GOOSEFS_* → storage_options mapping as the CLI.
• Form UX: dbConfigSetting honors ui_hidden_configs() and treats None defaults as empty strings so optional LanceDB fields don't break the form.
• Index param display: partition / PQ / HNSW params show only for the matching LanceDB index types.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: XuQianJin-Stars
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[XuQianJin-Stars]

/assign @XuanYang-cn

[XuQianJin-Stars]

Hi @XuanYang-cn, all checks have passed and there are no conflicts. Could you help review this PR when you get a chance? Thanks!

[yhmo]

vectordb_bench/backend/clients/lancedb/lancedb.py line:37
Medium ---- Enabling shared concurrent writes exposes the existing multi-fragment retry path: insert_embeddings() can commit an earlier LANCEDB_BATCH_SIZE chunk, fail on a later table.add(), return (0, error), and ConcurrentInsertRunner then retries the whole batch. With NUM_PER_BATCH > LANCEDB_BATCH_SIZE, this can duplicate IDs and corrupt benchmark counts. Please make the insert atomic/idempotent or return the committed prefix as non-retryable before setting thread_safe=True, and add an exact-row-count concurrency regression test.

[XuQianJin-Stars]

vectordb_bench/backend/clients/lancedb/lancedb.py line:37 Medium ---- Enabling shared concurrent writes exposes the existing multi-fragment retry path: insert_embeddings() can commit an earlier LANCEDB_BATCH_SIZE chunk, fail on a later table.add(), return (0, error), and ConcurrentInsertRunner then retries the whole batch. With NUM_PER_BATCH > LANCEDB_BATCH_SIZE, this can duplicate IDs and corrupt benchmark counts. Please make the insert atomic/idempotent or return the committed prefix as non-retryable before setting thread_safe=True, and add an exact-row-count concurrency regression test.

Thanks for the detailed analysis — agreed this was a real risk with thread_safe=True when NUM_PER_BATCH > LANCEDB_BATCH_SIZE.

Fixed by tracking committed rows across fragment table.add() calls. If a later add fails after a prefix was already committed, we now return (inserted, PartialInsertError(...)) (non_retryable=True) instead of (0, error), so ConcurrentInsertRunner will not retry the whole batch and duplicate IDs.

Also added regression coverage for the partial-commit path, ConcurrentInsertRunner non-retry behavior, and an exact row-count check under concurrent inserts.

[yhmo]

vectordb_bench/backend/clients/lancedb/config.py line:67
Low ---- This truthiness check makes an explicitly supplied empty mapping behave as “not supplied”. A REST/programmatic caller that uses storage_options={} to disable inherited settings will now pick up GOOSEFS_*/COS values from the process environment, contrary to the docstring that explicit options win. Please check explicit is not None so callers can intentionally select no options.

[XuQianJin-Stars]

vectordb_bench/backend/clients/lancedb/config.py line:67 Low ---- This truthiness check makes an explicitly supplied empty mapping behave as “not supplied”. A REST/programmatic caller that uses storage_options={} to disable inherited settings will now pick up GOOSEFS_*/COS values from the process environment, contrary to the docstring that explicit options win. Please check explicit is not None so callers can intentionally select no options.

Thanks for catching this. Fixed in the follow-up commit: the check is now if explicit is not None:, so an explicitly supplied empty mapping (storage_options={}) disables env inheritance as documented. Also added a regression test covering this case.