[Store] Prevent file-per-key path collisions#2984
Conversation
There was a problem hiding this comment.
Code Review
This pull request transitions the file-per-key storage layout to use stable digests generated via XXH3 128-bit hashing, replacing the legacy path resolution. It introduces legacy path fallback mechanisms in IsExist and BatchLoad to maintain backward compatibility with existing cache files, and adds validation to prevent collisions. The review feedback suggests wrapping struct_pb::from_pb in a try-catch block within ScanMeta to prevent corrupted files from crashing the metadata scanning process, and handling std::errc::no_such_file_or_directory gracefully in IsExist when checking file size to handle concurrent deletions robustly.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| KVEntry kv; | ||
| struct_pb::from_pb(kv, buf); |
There was a problem hiding this comment.
struct_pb::from_pb can throw an exception if the file content is corrupted or not a valid protobuf message. In ScanMeta, which scans the entire storage directory, an unhandled exception will cause the entire metadata scanning process to fail or crash the application. Wrapping it in a try-catch block and continuing the loop ensures robustness against individual corrupted files.
KVEntry kv;
try {
struct_pb::from_pb(kv, buf);
} catch (const std::exception& e) {
LOG(ERROR) << "Failed to decode file: " << p.string()
<< ", error: " << e.what();
continue;
}There was a problem hiding this comment.
Thanks. This pre-existing ScanMeta hardening is handled separately in #2987, with malformed-file coverage, so this PR stays focused on path collisions.
| std::error_code ec; | ||
| const auto size = fs::file_size(legacy_path, ec); | ||
| if (ec) { | ||
| return tl::make_unexpected(ErrorCode::FILE_READ_FAIL); | ||
| } |
There was a problem hiding this comment.
If fs::file_size fails because the file was concurrently deleted or does not exist (e.g., std::errc::no_such_file_or_directory), IsExist should return false instead of failing with ErrorCode::FILE_READ_FAIL. This makes the existence check more robust against concurrent evictions.
std::error_code ec;
const auto size = fs::file_size(legacy_path, ec);
if (ec) {
if (ec == std::errc::no_such_file_or_directory) {
return false;
}
return tl::make_unexpected(ErrorCode::FILE_READ_FAIL);
}There was a problem hiding this comment.
Handled in b63b817: IsExist() now treats a concurrently removed legacy file as absent, with regression coverage.
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Description
Fixes #2980.
Module
mooncake-store)Type of Change
How Has This Been Tested?
Test commands:
Test results (Ubuntu):
storage_backend_test: 74/74;utils_test: 12/12)Checklist
./scripts/code_format.shpre-commit run --all-filesand all hooks pass (touched-file hooks pass)AI Assistance Disclosure