struct KSUID

• KSUID
• Struct
• Value
• Object

Overview

KSUID stands for K-Sortable Unique IDentifier, a globally unique identifier used by Segment.

Distributed systems require unique identifiers to track events throughout their subsystems. Many algorithms for generating unique identifiers, like the Snowflake ID system, require coordination with a central authority. This is an unacceptable constraint in the face of systems that run on client devices, yet we still need to be able to generate event identifiers and roughly sort them for processing.

The KSUID optimizes this problem into a roughly sortable identifier with a high possibility space to reduce the chance of collision. KSUID uses a 32-bit timestamp with second-level precision combined with 128 bytes of random data for the "payload". The timestamp is based on the Unix epoch, but with its base shifted forward from 1970-01-01 00:00:00 UTC to 2014-05-13 16:53:20 UTC. This is to extend the useful life of the ID format to over 100 years.

Because KSUID timestamps use seconds as their unit of precision, they are unsuitable to tasks that require extreme levels of precision. If you need microsecond-level precision, a format like ULID may be more suitable for your use case.

KSUIDs are "roughly sorted". Practically, this means that for any given event stream, there may be some events that are ordered in a slightly different way than they actually happened. There are two reasons for this. Firstly, the format is precise to the second. This means that two events that are generated in the same second will be sorted together, but the KSUID with the smaller payload value will be sorted first. Secondly, the format is generated on the client device using its clock, so KSUID is susceptible to clock shift as well. The result of sorting the identifiers is that they will be sorted into groups of identifiers that happened in the same second according to their generating device.

See the canonical implementation for more information.

require "ksuid"

# Generate a random KSUID for the present time
KSUID.new

# Generate a random KSUID for a specific timestamp
KSUID.new(time: Time.utc - 3.hours)

# Parse a KSUID string that you have received
KSUID.from("0o5Fs0EELR0fUjHjbCnEtdUwQe3")

# Parse a KSUID byte slice that you have received
KSUID.from(Bytes.new(20, 255_u8))

Included Modules

• Comparable(KSUID)

Defined in:

Constant Summary

EPOCH = 14e8.to_u32

KSUID's epoch starts more recently so that the 32-bit number space gives a significantly higher useful lifetime of around 136 years from May 2014. This number (14e8) was picked to be easy to remember.

MAX = from(StaticArray(UInt8, TOTAL_SIZE).new(255_u8))

KSUID with maximum valid value (aWgEPTl1tmebfsQzFP4bxwgy80V).

MIN = from(StaticArray(UInt8, TOTAL_SIZE).new(0_u8))

KSUID with minimum valid value (000000000000000000000000000).

PAYLOAD_SIZE = 16

Payload is 16 bytes.

STRING_ENCODED_SIZE = 27

The length of a KSUID when string (base62) encoded.

TIMESTAMP_SIZE = 4

Timestamp is an UInt32.

TOTAL_SIZE = 20

KSUIDs are 20 bytes:

• 00-03 byte: UInt32 BE UTC #timestamp with custom epoch
• 04-19 byte: random #payload

VERSION = {{ (`shards version /srv/crystaldoc.info/github-Sija-ksuid.cr-v0.5.1/src/ksuid`).chomp.stringify }}

Constructors

• .from(string : String) : KSUID

  Converts a base62-encoded String into a KSUID.

• .from(bytes : Bytes) : KSUID

  Converts Bytes into a KSUID.

• .from(array : StaticArray(UInt8, TOTAL_SIZE)) : KSUID

  Converts StaticArray into a KSUID.

• .new(time : Time, payload : Bytes)

  Generates a new KSUID with given time and payload.

• .new(timestamp : Int, payload : Bytes)

  Generates a new KSUID with given timestamp and payload.

• .new(ctx : YAML::ParseContext, node : YAML::Nodes::Node)
• .new(time : Time = Time.utc)

  Generates a new KSUID with given time and random #payload.

• .new(pull : JSON::PullParser)

Instance Method Summary

• #<=>(other : KSUID)

  Compares the KSUID against other.

• #==(other : self)
• #hash(hasher)

  See Object#hash(hasher)

• #inspect(io : IO) : Nil

  Appends this struct's name and instance variables names and values to the given IO.

• #payload : StaticArray(UInt8, PAYLOAD_SIZE)

  The 16-byte random payload without the timestamp.

• #raw : String

  Returns the KSUID as a hex-encoded String.

• #timestamp : UInt32

  The timestamp portion of the KSUID as an UInt32 which is uncorrected for KSUID's special EPOCH.

• #to_json(json : JSON::Builder)
• #to_s(io : IO) : Nil

  Writes the KSUID as a base62-encoded String to io.

• #to_slice : Bytes

  Returns the KSUID as Bytes.

• #to_time : Time

  Returns the timestamp portion of the KSUID as a Time object.

• #to_yaml(yaml : YAML::Nodes::Builder)

Constructor Detail

Instance Method Detail

struct Point
  def initialize(@x : Int32, @y : Int32)
  end
end

p1 = Point.new 1, 2
p1.to_s    # "Point(@x=1, @y=2)"
p1.inspect # "Point(@x=1, @y=2)"