module Memo::Projection

Overview

Random projection operations for fast similarity filtering

Uses random orthogonal vectors to project high-dimensional embeddings into a low-dimensional space. Similar embeddings will have similar projections, enabling fast pre-filtering before full cosine similarity.

Extended Modules

Defined in:

memo/projection.cr

Constant Summary

K = 8

Number of projection dimensions (columns: proj_0 through proj_7)

Instance Method Summary

Instance Method Detail

def compute_projections(embedding : Array(Float64), proj_vectors : Array(Array(Float64))) : Array(Float64) #

Compute projections of an embedding onto projection vectors

Returns k dot products (one per projection vector)


[View source]
def dot_product(a : Array(Float64), b : Array(Float64)) : Float64 #

Compute dot product of two vectors


[View source]
def generate_orthogonal_vectors(dimensions : Int32, k : Int32 = K) : Array(Array(Float64)) #

Generate k random orthogonal unit vectors of given dimension

Uses Gram-Schmidt orthogonalization to ensure vectors are orthogonal. Each vector is normalized to unit length.


[View source]
def get_projection_vectors(db : DB::Database, service_id : Int64) : Array(Array(Float64)) | Nil #

Get projection vectors for a service

Returns nil if not found


[View source]
def get_projections(db : DB::Database, hash : Bytes) : Array(Float64) | Nil #

Get projections for an embedding

Returns nil if not found


[View source]
def normalize(vec : Array(Float64)) : Array(Float64) #

Normalize vector to unit length


[View source]
def projection_distance_squared(a : Array(Float64), b : Array(Float64)) : Float64 #

Compute squared Euclidean distance between two projection vectors

Used for fast filtering - smaller distance means more likely to be similar


[View source]
def store_projection_vectors(db : DB::Database, service_id : Int64, vectors : Array(Array(Float64))) #

Store projection vectors for a service


[View source]
def store_projections(db : DB::Database, hash : Bytes, projections : Array(Float64)) #

Store projections for an embedding


[View source]