Qdrant

Change log

Features :point_up:

• https://github.com/qdrant/qdrant/milestone/50 - TurboQuant 4-bit as a datatype of primary vector storage. Only store 4-bit quantized vectors and spare disk space on original vectors. [docs]
• https://github.com/qdrant/qdrant/pull/9669, https://github.com/qdrant/qdrant/pull/9684, https://github.com/qdrant/qdrant/pull/9950 - Unify definition of memory usage strategy for collection components. Use "memory": "cold" / "cached" / "pinned" to define memory behavior for each individual collection component. Allows for more fine-grained control over memory usage and performance. [docs]
• https://github.com/qdrant/qdrant/pull/9683 - Allow "match": {"prefix": "..."} in filter to match keywords by prefix, must be enabled in keyword index. [docs]
• https://github.com/qdrant/qdrant/pull/9661 - Per-query IDF corpus for sparse vector search [docs]
• https://github.com/qdrant/qdrant/pull/9899 - Slice filtering condition: sliced scroll / deterministic sampling [docs]
• https://github.com/qdrant/qdrant/pull/10035 - Global quota API [docs]
• https://github.com/qdrant/qdrant/pull/9338 - Add routing token for deterministic read routes [docs]

Improvements :point_left:

• https://github.com/qdrant/qdrant/pull/9113 - Use batched reads in vector and payload storage
• https://github.com/qdrant/qdrant/pull/9409 - Utilize io_uring for payload storage
• https://github.com/qdrant/qdrant/pull/9448 - Lower default update queue length, reduce from 1M to 200
• https://github.com/qdrant/qdrant/pull/9500 - Use Entry API to avoid redundant map double-lookups, improving performance
• https://github.com/qdrant/qdrant/pull/9332 - Enable single file mmap vector storage by default for immutable segments
• https://github.com/qdrant/qdrant/pull/9376 - Add option to explicitly disable BM25 stemmer; deprecate "none" hack
• https://github.com/qdrant/qdrant/pull/9457 - Performance improvements for facets
• https://github.com/qdrant/qdrant/pull/9431 - Respond HTTP 405 on cluster endpoints when in standalone mode
• https://github.com/qdrant/qdrant/pull/9662 - Restart consensus thread on failure instead of stopping permanently
• https://github.com/qdrant/qdrant/pull/9663 - Trace-log 'Missing request message' internal gRPC errors
• https://github.com/qdrant/qdrant/pull/9678 - Resolve filter-based update operations to point IDs before writing in WAL
• https://github.com/qdrant/qdrant/pull/9732 - Support snapshot recovery of streamable segments without nested files/ dir
• https://github.com/qdrant/qdrant/pull/9726 - Reduce memory usage for slow requests logging
• https://github.com/qdrant/qdrant/pull/9733 - Add time in gRPC responses
• https://github.com/qdrant/qdrant/pull/9644 - Set S3 user-agent on snapshot uploads
• https://github.com/qdrant/qdrant/pull/9749 - Lazy roaring flags bitmap and bool index counts
• https://github.com/qdrant/qdrant/pull/9778 - Dedup sub-hashes with AHashSet in immutable index, improving performance
• https://github.com/qdrant/qdrant/pull/9784 - Mark point deleted once instead of per value in immutable index, improving performance
• https://github.com/qdrant/qdrant/pull/9660 - Speed up TurboQuant Hadamard rotation by ~3x
• https://github.com/qdrant/qdrant/pull/9835 - Single inverse rotation in TurboQuant symmetric L1 scoring
• https://github.com/qdrant/qdrant/pull/9811 - Align scalar f16 metrics precision with SIMD thresholds
• https://github.com/qdrant/qdrant/pull/9847 - Improve error message for non-string key in filter conditions
• https://github.com/qdrant/qdrant/pull/9871 - Compact stored bitmask for on-disk field index deleted masks
• https://github.com/qdrant/qdrant/pull/9740, https://github.com/qdrant/qdrant/pull/9845 - Use batching in filter condition checker
• https://github.com/qdrant/qdrant/pull/9984 - Evict page cache of files that are mapped twice
• https://github.com/qdrant/qdrant/pull/10026 - Serialize snapshot recoveries of the same shard
• https://github.com/qdrant/qdrant/pull/9891 - Report effective (cgroup) CPU, RAM and disk in telemetry
• https://github.com/qdrant/qdrant/pull/9721 - Use jemalloc 5.3.1 to support page sizes larger than 4KB on aarch64 systems

Bug Fixes :facepunch:

• https://github.com/qdrant/qdrant/pull/9395 - Fixes for resharding abort logic consistency
• https://github.com/qdrant/qdrant/pull/9377 - Read deferred source point in copy-on-write move
• https://github.com/qdrant/qdrant/pull/9424 - Fix data loss when a payload index drop races the background segment flush
• https://github.com/qdrant/qdrant/pull/7849 - Fix abort resharding live-lock
• https://github.com/qdrant/qdrant/pull/9501 - Buffer multi-dense offsets store to prevent reload corruption
• https://github.com/qdrant/qdrant/pull/9401 - Fix empty min_should with non-zero min_count matching everything
• https://github.com/qdrant/qdrant/pull/9536 - Fix optimizer data race, defer source segment destruction until durable
• https://github.com/qdrant/qdrant/pull/9518 - Fix scalar L2 quantized score shift
• https://github.com/qdrant/qdrant/pull/9537 - Fix snapshot WAL clocks data race
• https://github.com/qdrant/qdrant/pull/9604, https://github.com/qdrant/qdrant/pull/9637 - Avoid allocator abort on huge limit in random search/sample/scroll
• https://github.com/qdrant/qdrant/pull/9610 - Fix for Point Group Search query with large limits or group sizes cause panics
• https://github.com/qdrant/qdrant/pull/9654 - Fix reinit failing with "removed all voters" on a removed peer
• https://github.com/qdrant/qdrant/pull/9671 - Fix optimizer recreation dropping sibling shard restarts on error
• https://github.com/qdrant/qdrant/pull/9641 - Unblock optimizer after deleting a named vector
• https://github.com/qdrant/qdrant/pull/9686 - Fix cancelled snapshot recovery dirty marker being left behind
• https://github.com/qdrant/qdrant/pull/9688 - Fix /readyz of reinitialized peer waiting on a foreign consensus
• https://github.com/qdrant/qdrant/pull/9677 - Preserve empty result shape for zero-limit batches
• https://github.com/qdrant/qdrant/pull/9723 - Fix drive letter parsing in Windows snapshot mappings
• https://github.com/qdrant/qdrant/pull/9774 - Fix /readyz false positive on freshly bootstrapped peer
• https://github.com/qdrant/qdrant/pull/9767 - Fix payload index inconsistency after restart: flush-before-build
• https://github.com/qdrant/qdrant/pull/9806 - Stamp payload storage version on append-only clone paths
• https://github.com/qdrant/qdrant/pull/9801 - Fix use-after-free in u8-quantized vector reads on owning storages
• https://github.com/qdrant/qdrant/pull/9638 - Return error for missing point IDs in Query API recommend
• https://github.com/qdrant/qdrant/pull/9785 - Reset first_voter and prune address book on first-peer --reinit
• https://github.com/qdrant/qdrant/pull/9759 - Advance newest clocks for WAL-replay tail on load
• https://github.com/qdrant/qdrant/pull/9660 - Count trailing deleted offset on mmap sparse reload
• https://github.com/qdrant/qdrant/pull/9851 - Clamp WAL replay target to the truncated prefix on shard load
• https://github.com/qdrant/qdrant/pull/9882 - Fix resharding, on queries filter shards on all shard selectors
• https://github.com/qdrant/qdrant/pull/9868 - Fix WAL data race, flush after copy with Segment::copy_to_path
• https://github.com/qdrant/qdrant/pull/9928 - Clean up stale shard transfers when applying consensus snapshot
• https://github.com/qdrant/qdrant/pull/9930 - Honor applied_seq during WAL replay only under prevent_unoptimized
• https://github.com/qdrant/qdrant/pull/10005 - Fix TurboQuant quantized vector layout alignment
• https://github.com/qdrant/qdrant/pull/9992 - Flush after pref-illing deleted vectors
• https://github.com/qdrant/qdrant/pull/10059 - Clear joint consensus fields on first peer reinit
• https://github.com/qdrant/qdrant/pull/9105, https://github.com/qdrant/qdrant/pull/9410, https://github.com/qdrant/qdrant/pull/9317 - Fixes for handling of creation and deletion of named vectors
• https://github.com/qdrant/qdrant/pull/10085 - Fix path traversal vulnerability in S3 based snapshots

Deprecations :point_down:

• https://github.com/qdrant/qdrant/pull/9982 - Remove deprecated search endpoints from OpenAPI, deprecate them in gRPC
• https://github.com/qdrant/qdrant/pull/10020 - Make deprecated on_disk_payload optional in the REST schema
• https://github.com/qdrant/qdrant/pull/10035 - Deprecates max_resident_memory_percent of strict mode in favor of global quota API

Web UI :open_hands:

• https://github.com/qdrant/qdrant-web-ui/milestone/1 - Significantly more performant vectors visualization and dimensionality reduction using WASM implementation of UMAP and WebGL rendering
• https://github.com/qdrant/qdrant-web-ui/releases/tag/v0.2.15 - UI for managing payload indexes and their configuration
• https://github.com/qdrant/qdrant-web-ui/pull/428 - UI for managing global resource quotas
• https://github.com/qdrant/qdrant-web-ui/pull/425 - Display for resharding progress
• https://github.com/qdrant/qdrant-web-ui/pull/400 - Multi delete feature for collections
• https://github.com/qdrant/qdrant-web-ui/pull/424 - Display and editing for collection metadata

Change log

Bug Fixes

• https://github.com/qdrant/qdrant/pull/9882 - Fix query errors when using shard keys while resharding

Change log

Improvements

• https://github.com/qdrant/qdrant/pull/9282 - Log slow operations during shard WAL recovery
• https://github.com/qdrant/qdrant/pull/9137 - Clear cache of ID tracker after building a segment
• https://github.com/qdrant/qdrant/pull/9138 - Don't rebuild payload index if changing on_disk flag

Bug Fixes

• https://github.com/qdrant/qdrant/pull/9285 - Fix potential infinite loop in optimizer when using multi vectors with prevent_unoptimized
• https://github.com/qdrant/qdrant/pull/9217 - Clean up unfinished segment optimizations on cancellation, don't load them on restart
• https://github.com/qdrant/qdrant/pull/9260 - Fix MatchAny with an empty list being rejected on integer index
• https://github.com/qdrant/qdrant/pull/9239 - Add timeout to shard snapshot streaming endpoint, drop connections not actively read from
• https://github.com/qdrant/qdrant/pull/9215 - Fix abort transfer with resharding not being idempotent
• https://github.com/qdrant/qdrant/pull/9237 - Fix reporting old progress in new snapshot transfers
• https://github.com/qdrant/qdrant/pull/9226 - Fix WAL lock error on Android platforms

Security

• https://github.com/qdrant/qdrant/pull/9254 - Fix REST auth whitelist bypass on specially crafted paths, resolve route before authorizing
• https://github.com/qdrant/qdrant/pull/9268 - Fix out of bound heap read with malicious snapshot by rejecting incorrect length

Change log

Improvements

• https://github.com/qdrant/qdrant/pull/8988 - Refactor quantized multi-vector scorers for io_uring support
• https://github.com/qdrant/qdrant/pull/9058 - Validate vector dimensions before WAL write for async upserts

Bug Fixes

• https://github.com/qdrant/qdrant/pull/8990 - Notify pending consensus ops on snapshot apply
• https://github.com/qdrant/qdrant/pull/9054 - Fix indexed integer range filter with float values
• https://github.com/qdrant/qdrant/pull/9055 - Fix {match: {except: []}} returning zero results with payload index
• https://github.com/qdrant/qdrant/pull/9070 - Fix empty vector panic
• https://github.com/qdrant/qdrant/pull/9099 - Fix TurboQuant heap memory under-reporting
• https://github.com/qdrant/qdrant/pull/9107 - Do not use unnecessary sequential mmaps
• https://github.com/qdrant/qdrant/pull/9014 - Fix resharding cleanup data race with update queue
• https://github.com/qdrant/qdrant/pull/9120 - Use cluster default shard transfer method for fallback
• https://github.com/qdrant/qdrant/pull/9121 - Recreate workers/optimizers async to not block consensus
• https://github.com/qdrant/qdrant/pull/9110 - Restart optimizer on conflicting vector schema
• https://github.com/qdrant/qdrant/pull/9122 - Do not remove shard for snapshot transfer receiver
• https://github.com/qdrant/qdrant/pull/9125 - Fix snapshot transfer cancellation on external malformed requests
• https://github.com/qdrant/qdrant/pull/9116 - Fix deleted mask race on optimization and snapshot creation

Security

• https://github.com/qdrant/qdrant/pull/9031 - Authorize request before we accept snapshot file upload

Change log

Features 🏎️

• milestone!48 - Add TurboQuant quantization variant, 8x vector compression without the recall tax
• https://github.com/qdrant/qdrant/pull/8605 - Add API to create and delete named vectors in existing collection
• https://github.com/qdrant/qdrant/pull/8606 - Deep memory reporting, show memory usage breakdown for storage components
• https://github.com/qdrant/qdrant/pull/8714 - Add low memory mode, force open everything on disk to minimize out of memory crash on startup
• https://github.com/qdrant/qdrant/pull/8715 - Add strict mode parameter to reject updates when memory usage is high (max_resident_memory_percent)

Improvements 📏

• https://github.com/qdrant/qdrant/pull/8790, https://github.com/qdrant/qdrant/pull/8769 - Use dynamic CPU pool for search workers, improve search performance when there's high IO wait
• https://github.com/qdrant/qdrant/pull/8526, https://github.com/qdrant/qdrant/pull/8656 - Use operation size based batching in shard transfers to achieve higher throughput
• https://github.com/qdrant/qdrant/pull/8673 - Reduce memory usage of immutable geo index by 7x
• https://github.com/qdrant/qdrant/pull/8529 - Fully remove RocksDB support, simplifying storage handling
• https://github.com/qdrant/qdrant/pull/8768 - Don't stall update queue when waiting for deferred points
• https://github.com/qdrant/qdrant/pull/8944 - Use default timeout of 60 seconds to update requests, clean up long running update requests
• https://github.com/qdrant/qdrant/pull/8767 - When collection update triggers optimizer recreation, clear any optimizer errors to allow recovery without restart
• https://github.com/qdrant/qdrant/pull/8784 - Use snapshot based shard transfers for much faster recovery if WAL delta cannot be resolved
• https://github.com/qdrant/qdrant/pull/8782 - On snapshot transfer, first clear shard data on receiver to prevent out of disk crash
• https://github.com/qdrant/qdrant/pull/8689 - Clean temporary directories before loading collections, may prevent out of disk crash on recovery
• https://github.com/qdrant/qdrant/pull/8830 - Don't hold shard holder lock during search to prevent blocking various other operations
• https://github.com/qdrant/qdrant/pull/8626, https://github.com/qdrant/qdrant/pull/8635 - Record API method path in audit log events
• https://github.com/qdrant/qdrant/pull/8402 - Report tracing ID in audit log events
• https://github.com/qdrant/qdrant/pull/8636 - Report audit log configuration and data size in telemetry and metrics
• https://github.com/qdrant/qdrant/pull/8640 - Optimize cosine normalization on ARM CPUs (SIMD)
• https://github.com/qdrant/qdrant/pull/8571 - Skip audit logging on telemetry endpoints
• https://github.com/qdrant/qdrant/pull/8693 - Make HNSW sub graph building deterministic
• https://github.com/qdrant/qdrant/pull/8712, https://github.com/qdrant/qdrant/pull/8708, https://github.com/qdrant/qdrant/pull/8706 - Report various structures in deterministic order
• https://github.com/qdrant/qdrant/pull/8932 - Link to agentic skills in startup message

Bug Fixes 🪺

• https://github.com/qdrant/qdrant/pull/8728 - Fix stop words always being lowercased over gRPC even if disabled
• https://github.com/qdrant/qdrant/pull/8719 - Fix datetime parsing for YYYY-MM-DDTHH:MM format
• https://github.com/qdrant/qdrant/pull/8734 - Fix IsEmpty condition on freshly rebuilt null index
• https://github.com/qdrant/qdrant/pull/8939 - Fix nested MatchTextAny not using full-text index
• https://github.com/qdrant/qdrant/pull/8646 - Fix geo circle hash bound checks
• https://github.com/qdrant/qdrant/pull/8938 - Fix phrase matching crossing string-array element boundaries
• https://github.com/qdrant/qdrant/pull/8762 - Fix hard panic when there are multiple validation errors
• https://github.com/qdrant/qdrant/pull/8194 - Disallow sparse vectors with an empty name
• https://github.com/qdrant/qdrant/pull/8832 - Fix GPU not being used for HNSW without main graph
• https://github.com/qdrant/qdrant/pull/8691 - Fix infinite loop in HNSW sub graph building
• https://github.com/qdrant/qdrant/pull/8577 - Fix panic when snapshot path contains non-UTF-8 characters
• https://github.com/qdrant/qdrant/pull/8293 - Fix hard panic on corrupt collection alias file, return error instead
• https://github.com/qdrant/qdrant/pull/8574 - Fix disk IO not always being measured in map index
• https://github.com/qdrant/qdrant/pull/8645 - Fix chunk capacity reservation to allocate more efficiently
• https://github.com/qdrant/qdrant/pull/8651 - Fix payload index flushers absorbing cancellations as errors
• https://github.com/qdrant/qdrant/pull/8474 - Fix snapshot files not cleaned up on snapshot restore failure
• https://github.com/qdrant/qdrant/pull/8654 - Fix clearing page cache not being reliable, add missing candidates
• https://github.com/qdrant/qdrant/pull/8542 - Fix map index listing modified file as immutable
• https://github.com/qdrant/qdrant/pull/8847 - Fix data race causing disk watcher not to be woken up causing timeouts
• https://github.com/qdrant/qdrant/pull/8789, https://github.com/qdrant/qdrant/pull/8917 - Make resharding operations idempotent, fix cluster state corruption on crash
• https://github.com/qdrant/qdrant/pull/8831 - Fix slow request logging for read operations with shard level errors
• https://github.com/qdrant/qdrant/pull/8756 - Don't allocate async search workers we don't use reducing thread count
• https://github.com/qdrant/qdrant/pull/8596 - Don't write to old shards on resharding down after write hashring commit, fix missing shard errors
• https://github.com/qdrant/qdrant/pull/8543 - Fix edge sparse vector search panic on score post processing
• https://github.com/qdrant/qdrant/pull/8588 - Fix various Raft consensus correctness bugs upstream

Security 🕵️

• https://github.com/qdrant/qdrant/pull/8676 - Enforce API key/JWT authentication on internal gRPC endpoints
• https://github.com/qdrant/qdrant/pull/8628 - Add config option to disable snapshot restore from URL
• https://github.com/qdrant/qdrant/pull/8619 - Bump TLS dependencies to resolve security vulnerabilities

Web UI 🍱

• https://github.com/qdrant/qdrant-web-ui/pull/372 - Add memory and disk inspector, view usage breakdown per storage component
• https://github.com/qdrant/qdrant-web-ui/pull/370 - Add high contrast theme
• https://github.com/qdrant/qdrant-web-ui/pull/369 - Render inline documentation in collection info panel
• https://github.com/qdrant/qdrant-web-ui/pull/362 - Add refresh button to some collection views
• https://github.com/qdrant/qdrant-web-ui/pull/373, https://github.com/qdrant/qdrant-web-ui/pull/377 - Fix search quality latency measurement and use better naming

Change log

Improvements

• milestone#46 - Defer point updates, efficiently apply and optimize points with prevent_unoptimized=true
• https://github.com/qdrant/qdrant/pull/8188 - Make Gridstore flushes non-blocking to reduce search tail latencies
• https://github.com/qdrant/qdrant/pull/8235 - Improve performance of filtered search in case of singular payload value
• https://github.com/qdrant/qdrant/pull/8402 - Add request tracing ID into audit log
• https://github.com/qdrant/qdrant/pull/8460 - Propagate WAL errors instead of panicking during loading of shards
• https://github.com/qdrant/qdrant/pull/8301 - Allow peer to bootstrap with used URI if empty

Bug fixes

• https://github.com/qdrant/qdrant/pull/8177 - GPU: fix raw Vulkan name pointer type
• https://github.com/qdrant/qdrant/pull/8193 - Fix creation of uninitialized shard key with replication factor > 1, fixing tiered multi-tenancy workflow
• https://github.com/qdrant/qdrant/pull/8217 - Prevent min_should panic on large amount of conditions
• https://github.com/qdrant/qdrant/pull/8179 - Fix for restore of cluster snapshots which creates unnecessary replicas in Partial state
• https://github.com/qdrant/qdrant/pull/8220 - Fix Server disconnected without sending a response error while performing concurrent ingestion using
• https://github.com/qdrant/qdrant/pull/8341 - Security patch to force snapshot recovery from snapshot directory only
• https://github.com/qdrant/qdrant/pull/8373 - Fix lock for collection-level operations during shard transfer
• https://github.com/qdrant/qdrant/pull/8438 - Do not fail on collection-level operations with dummy shard
• https://github.com/qdrant/qdrant/pull/8455 - Fix WAL reading issue introduced in 1.17.0
• https://github.com/qdrant/qdrant/pull/8454 - Fix another panic in WAL replay
• https://github.com/qdrant/qdrant/pull/8475 - Fix for WAL delta transfer: reject truncated recovery points before equal-pruning
• https://github.com/qdrant/qdrant/pull/8514 - Fix incorrect warning in geo-index
• https://github.com/qdrant/qdrant/pull/8374 - Prevent locking shard holder for a long time in slow stream records transfers
• https://github.com/qdrant/qdrant/pull/8480 - Fix panic in optimizer logging
• https://github.com/qdrant/qdrant/pull/8449 - Fix panic in chunked vector storage

Preview

This features are not officially released yet, but available in the build:

• https://github.com/qdrant/qdrant/pull/8214 - Per-collection metrics in Prometheus
• https://github.com/qdrant/qdrant/pull/8498 - API for reading audit log entries
• https://github.com/qdrant/qdrant/pull/8469 - Maximum batch size config in strict mode

Edge

• Rust Crate for Qdrant Edge v0.6.0 - https://crates.io/crates/qdrant-edge
• Python Package for Qdrant Edge v0.6.0 - https://pypi.org/project/qdrant-edge-py/

Change log

Features 🏋️

• milestone#38 - Relevance Feedback (docs)
• milestone#44 - API for detailed report on optimization progress and stages (docs)
• milestone#40 - API for aggregated telemetry of the whole cluster (docs)
• milestone#43 - Unlimited update queue to gracefully smooth update spikes (docs)
• https://github.com/qdrant/qdrant/pull/8071 - Add Audit Access Logging (docs)
• https://github.com/qdrant/qdrant/pull/8063 - Add Weighted RRF (docs)
• https://github.com/qdrant/qdrant/pull/7643 - Add config option to control update throughput and prevent unoptimized searches (docs)
• https://github.com/qdrant/qdrant/pull/7929 - Add configurable read fan-out delay for dealing with tail latency in distributed clusters (docs)
• https://github.com/qdrant/qdrant/pull/7963 - For upserts, add update_mode parameter to either upsert, update or insert (docs)
• https://github.com/qdrant/qdrant/pull/7835 - Add secondary API key configuration for zero downtime key rotation in distributed clusters
• https://github.com/qdrant/qdrant/pull/7838 - Add dedicated HTTP port for /metrics endpoint for internal monitoring
• https://github.com/qdrant/qdrant/pull/7615 - Add API to list shard keys (docs)

Improvements 🤸

• https://github.com/qdrant/qdrant/pull/7802 - Improve timeout handling on read operations
• https://github.com/qdrant/qdrant/pull/7750 - Improve timeout handling in update operations, prevent shard failures in case of timed out updates after WAL
• https://github.com/qdrant/qdrant/pull/8025 - Recover snapshot without creating intermediate files, greatly improves recovery time and disk usage
• https://github.com/qdrant/qdrant/pull/8059 - Recover snapshots directly into target file system to avoid expensive file movements
• https://github.com/qdrant/qdrant/pull/7883 - Flush after snapshot unpack with syncfs to persist a large number of files much more efficiently
• https://github.com/qdrant/qdrant/pull/8072 - Don't lock shard holder structure during creation of a snapshot, previously blocking shard level operations
• https://github.com/qdrant/qdrant/pull/8166 - Add timeout to snapshot downloads, abort if connection gets stuck for more than a minute
• https://github.com/qdrant/qdrant/pull/8007, https://github.com/qdrant/qdrant/pull/8056 - Improve segments locking approach to minimize lock contention
• https://github.com/qdrant/qdrant/pull/8105 - Limit number of parallel updates on a shard to 64 to prevent order tracking overhead
• https://github.com/qdrant/qdrant/pull/8169 - Reduce locking in Gridstore to lower search tail latencies
• https://github.com/qdrant/qdrant/pull/8164 - Actively free cache memory for closed WAL segments to reduce memory pressure
• https://github.com/qdrant/qdrant/pull/7952 - Disable in-place payload updates on unindexed fields, improve immutability guarantees of indexed segments improving partial snapshots
• https://github.com/qdrant/qdrant/pull/7887 - Add ability to disable extra HNSW links construction for specific payload indices (docs)
• https://github.com/qdrant/qdrant/pull/7971 - Enable missing option for vector storage to populate single-file mmap
• https://github.com/qdrant/qdrant/pull/7928 - Enable io_uring when reading batch of vectors
• https://github.com/qdrant/qdrant/pull/7919 - Improve error message for datetime parse failures
• https://github.com/qdrant/qdrant/pull/8053 - Allow to configure load concurrency for collections, shards and segments
• https://github.com/qdrant/qdrant/pull/7809 - Add more convenient way to provide API-keys for external inference providers (docs)
• https://github.com/qdrant/qdrant/pull/8093 - Don't lock WAL during serialization of new updates, which was costly for large operations
• https://github.com/qdrant/qdrant/pull/7834 - Extend WAL retention when replicas are dead, prevent full shard transfers in case of peer failures
• https://github.com/qdrant/qdrant/pull/7565 - Disable old shard key format deprecated in 1.15.0
• https://github.com/qdrant/qdrant/pull/8125 - Skip building extra HNSW links for deleted vectors
• https://github.com/qdrant/qdrant/pull/8163 - Improve search result processing to use less CPU with a high search limit
• https://github.com/qdrant/qdrant/pull/8175 - Use less allocations for HNSW plain filtered search

Bug Fixes 🤹

• https://github.com/qdrant/qdrant/pull/7850 - Fix flush ordering to follow segment dependencies, prevents dataloss by CoW on flush interruption
• https://github.com/qdrant/qdrant/pull/8103 - Fix data race in stream records transfer potentially missing ongoing updates
• https://github.com/qdrant/qdrant/pull/7983 - Fix interlocking problem on creation of payload index
• https://github.com/qdrant/qdrant/pull/7999 - Fix interlocking problem on collection-level update operations
• https://github.com/qdrant/qdrant/pull/8131 - Fix deadlock during snapshot with concurrent updates
• https://github.com/qdrant/qdrant/pull/8128 - Fix gRPC/HTTP2 too_many_internal_resets error due to how we internally cancel ongoing requests
• https://github.com/qdrant/qdrant/pull/8019 - Improve handling of HTTP2 channels closing in connection pool
• https://github.com/qdrant/qdrant/pull/8104 - Fix data race in WAL and shard clocks snapshot, ensure they remain consistent
• https://github.com/qdrant/qdrant/pull/7961 - Fix using incorrect versions in partial snapshot manifest construction
• https://github.com/qdrant/qdrant/pull/8095 - Fix incorrect internal protocol usage for shard snapshot transfers
• https://github.com/qdrant/qdrant/pull/7950 - Fix integer overflow in query batch when using high limits
• https://github.com/qdrant/qdrant/pull/7972 - Fix search aggregator panic with limit 0
• https://github.com/qdrant/qdrant/pull/8100 - Fix round floats not used in integer index, JSON doesn't distinguish between integers and floats
• https://github.com/qdrant/qdrant/pull/8097 - Fix score_threshold not being used in score boosting queries
• https://github.com/qdrant/qdrant/pull/7877 - Fix Corrupted ID tracker mapping storage bug when disk is full
• https://github.com/qdrant/qdrant/pull/7944 - Fix gRPC API response status counting in telemetry and metrics
• https://github.com/qdrant/qdrant/pull/7857 - Fix total count in progress tracker for replicate points with filter
• https://github.com/qdrant/qdrant/pull/7856 - Fix creation of payload index in empty collection using user-defined sharding
• https://github.com/qdrant/qdrant/pull/8099 - Fix ignoring CA certs for internal requests if configured
• https://github.com/qdrant/qdrant/pull/8176 - Add missing timeout parameter to some endpoints

Web UI :bento:

• https://github.com/qdrant/qdrant-web-ui/pull/345 - Detailed visualization of optimization progress
• https://github.com/qdrant/qdrant-web-ui/pull/334 - Create collection dialog now previews the exact command
• https://github.com/qdrant/qdrant-web-ui/pull/341 - Buttons for resharding control
• https://github.com/qdrant/qdrant-web-ui/pull/344 - Points filter and search bar restored and improved

Qdrant Edge :hocho:

Qdrant Edge is an in-process version of Qdrant, which shares the same internals, storage format, and points API as the server version, but designed to work locally. Qdrant Edge is compatible with server version and it can read shard snapshots created by server version of Qdrant. More documentation available here.

Deprecations :warning:

• Starting from v1.17.0 Qdrant changes response format for vector fields in gRPC interface. All official Qdrant clients should be already adopted to this change, so please make sure you upgrade your client libraries and check that you are not using deprecated fields. More info: https://github.com/qdrant/qdrant/pull/7183

• Upcoming deprecations:

  • In Qdrant v1.18.x all deprecated search methods will be completely removed and won't be available even from old client libraries.
  • In Qdrant v1.17.x we will completely remove RocksDB support in favor of gridstore, that means that direct upgrade from v1.15.x into v1.17.x won't be possible. Please follow upgrade instructions and upgrade one minor version at a time to avoid unsupported storage errors. Note that Qdrant Cloud infrastructure automatically generates a proper upgrade steps, so you don't have to worry about that.

Change log

Improvements

• https://github.com/qdrant/qdrant/pull/7755, https://github.com/qdrant/qdrant/pull/7588 - Respect search and point retrieve timeout when trying to access segments
• https://github.com/qdrant/qdrant/pull/7685 - Respect telemetry timeout when fetching shard statistics
• https://github.com/qdrant/qdrant/pull/7715 - Log snapshot download duration and speed

Bug fixes

• https://github.com/qdrant/qdrant/pull/7787, https://github.com/qdrant/qdrant/pull/7791 - Fix WAL delta transfer corrupting replica after a previous full transfer was aborted
• https://github.com/qdrant/qdrant/pull/7801, https://github.com/qdrant/qdrant/pull/7805 - Fix flush losing changes on transient disk IO errors, potentially corrupting data
• https://github.com/qdrant/qdrant/pull/7792 - Fix incorrectly aborting shard transfers when dropping unrelated shard
• https://github.com/qdrant/qdrant/pull/7741 - Fix flush error in Gridstore, potentially corrupting data when quickly alternating inserts/deletes
• https://github.com/qdrant/qdrant/pull/7702 - Fix flush data race in Gridstore, potentially corrupting data when storage is cleared in parallel
• https://github.com/qdrant/qdrant/pull/7759, https://github.com/qdrant/qdrant/pull/7782 - Fix handling of collection names with weird characters, breaking snapshot transfers for example
• https://github.com/qdrant/qdrant/pull/7788 - Fix snapshot metrics not always reporting when zero (snapshot_{creation,recovery}_running, snapshot_created_total)
• https://github.com/qdrant/qdrant/pull/7783 - Fix incorrectly reporting optimization errors in telemetry on panic
• https://github.com/qdrant/qdrant/pull/7765 - Fix slow shutdown on SIGINT when optimizations are running
• https://github.com/qdrant/qdrant/pull/7690 - Fix Qdrant not building on Windows ARM64
• https://github.com/qdrant/qdrant/pull/7683 - Keep RocksDB support until 1.18.0 in development builds

Change log

Improvements

• https://github.com/qdrant/qdrant/pull/7607 - Improve request timeout handling for telemetry and metrics
• https://github.com/qdrant/qdrant/pull/7623 - Add user agent to HTTP requests sent by Qdrant server

Bug fixes

• https://github.com/qdrant/qdrant/pull/7674 - Fix critical WAL bug that could break consensus or cause data corruption on restart
• https://github.com/qdrant/qdrant/pull/7684 - Fix consensus crash when applying consensus snapshot with non-replicated collection
• https://github.com/qdrant/qdrant/pull/7620 - Fix panic during search on segments with empty HNSW graph
• https://github.com/qdrant/qdrant/pull/7629, https://github.com/qdrant/qdrant/pull/7640, https://github.com/qdrant/qdrant/pull/7673 - Fix shard resource cleanup when shard is replaced, prevent deadlock on small CPUs
• https://github.com/qdrant/qdrant/pull/7621, https://github.com/qdrant/qdrant/pull/7626 - Fix payload index storage still flushing after removal, fixing data corruption and IO errors on Windows
• https://github.com/qdrant/qdrant/pull/7624, https://github.com/qdrant/qdrant/pull/7627 - Fix Gridstore storage still flushing after wipe, fixing data corruption and IO errors
• https://github.com/qdrant/qdrant/pull/7614, https://github.com/qdrant/qdrant/pull/7618 - Fix Docker/WSL on Windows with bind mount corrupting storage
• https://github.com/qdrant/qdrant/pull/7678 - Fix collections_vector_total metric reporting -0.0 if there are no vectors
• https://github.com/qdrant/qdrant/pull/7649 - Also report collection_indexed_only_excluded_points metric if zero

Change log

Improvements

• https://github.com/qdrant/qdrant/pull/7514, https://github.com/qdrant/qdrant/pull/7572 - Make batch queries up to 3 times faster on full scans by reading each point only once
• https://github.com/qdrant/qdrant/pull/7551 - Actively migrate vector, payload and payload index storage from RocksDB into Gridstore on startup for better and more predictable performance
• https://github.com/qdrant/qdrant/pull/7579 - Add 60s internal timeout for telemetry/metrics endpoints to prevent long hanging tasks
• https://github.com/qdrant/qdrant/pull/7557 - Add validation to restart shard transfer operation
• https://github.com/qdrant/qdrant/pull/7446 - Defer Gridstore flushing to make flushing behavior consistent with all other storage components
• https://github.com/qdrant/qdrant/pull/7580 - Improve consensus WAL compaction logging to aid debugging
• https://github.com/qdrant/qdrant/pull/7598 - Make timeout for inference requests user configurable

Bug fixes

• https://github.com/qdrant/qdrant/pull/7564 - Fix panic at startup on old clusters with user defined sharding, if not updated to Qdrant 1.15.5 first
• https://github.com/qdrant/qdrant/pull/7577 - Fix breaking Raft by killing node at specific time during consensus snapshot, preventing potential crash loop
• https://github.com/qdrant/qdrant/pull/7587 - Fix corrupting WAL with broken flush edge case after WAL is cleared or truncated
• https://github.com/qdrant/qdrant/pull/7570 - Fix incorrect rescoring default on mutable segments when using binary quantization
• https://github.com/qdrant/qdrant/pull/7569, https://github.com/qdrant/qdrant/pull/7575 - Spawn search/update tasks on the correct runtime, significantly reduce number of general/actix threads
• https://github.com/qdrant/qdrant/pull/7558 - Fix data race in shard transfers, wait on transfer to start before initializing shards
• https://github.com/qdrant/qdrant/pull/7556 - Fix incorrect log message when failing to read lock segment for some time