Skip to content

Add public API for computing partition assignment from a key - #489

Open
dskliarov wants to merge 1 commit into
lovoo:masterfrom
dskliarov:add-partition-key-api
Open

Add public API for computing partition assignment from a key#489
dskliarov wants to merge 1 commit into
lovoo:masterfrom
dskliarov:add-partition-key-api

Conversation

@dskliarov

Copy link
Copy Markdown

Summary

Goka's own partition assignment logic (View.hash()/Processor.hash()) is unexported in goka. In case if the client required to detect partition id by the key, he has to create his own version of the consistent hashing. There is a high possibility, that the logic could be changed and hashing calculation would silently diverge.

Adds PartitionForKey(key []byte, numPartitions int32, hasher func() hash.Hash32) (int32, error), a new exported function that computes the partition a key would be assigned to, using the same hash-and-mod algorithm goka already uses internally in Processor.hash and View.hash.

This lets external callers predict partition assignment (e.g. for routing, sharding-aware batching, or debugging) without needing a running View or Processor instance.

Changes

  • partition.go (new): PartitionForKey — validates numPartitions > 0, hashes the key with the provided hasher factory, and returns hash % numPartitions.
  • partition_test.go (new): covers basic range/determinism checks, single-partition behavior, empty keys, invalid partition counts, parity with
    sarama.NewCustomHashPartitioner, parity with the previous internal View.hash logic, and custom hasher usage.
  • processor.go: Processor.hash now delegates to PartitionForKey instead of inlining the hash logic.
  • view.go: View.hash now delegates to PartitionForKey instead of inlining the hash logic; drops the now-unused errors import.

Behavior notes

  • Same algorithm as before (FNV hash via DefaultHasher(), or whatever hasher was configured via WithHasher/WithViewHasher/WithEmitterHasher) — this is a refactor of
    the internal call sites, not a behavior change for existing users.
  • Error messages changed slightly ("can't hash with 0 partitions" / "no partitions found" → "invalid partition count: %d"); nothing in the codebase asserts on the
    old strings.
  • Callers must supply the same numPartitions and hasher that the live Processor/View uses — PartitionForKey doesn't read from a running instance, so a mismatch
    (e.g. wrong partition count, or DefaultHasher() when a custom hasher was configured) will silently produce a different partition than the real one.

Testing

  • partition_test.go verifies PartitionForKey matches both the sarama hash partitioner and the previous internal hashing logic across multiple keys and partition
    counts.
  • Existing processor.go/view.go test suites (view_test.go, etc.) continue to exercise the refactored hash methods unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant