class Cadmium::Glove::Corpus

Included Modules

Defined in:

glove/corpus.cr

Constructors

Class Method Summary

Instance Method Summary

Constructor Detail

def self.new(pull : JSON::PullParser) #

[View source]
def self.new(text, window = nil, min_count = nil, **parser_options) #

Create a new Glove::Corpus


[View source]

Class Method Detail

def self.build(text, window = nil, min_count = nil, **parser_options) #

Convenience method for creating an instance and building the token count and pairs.


[View source]

Instance Method Detail

def build_tokens #

Builds the token count, token index, and token pairs


[View source]
def count #

Hash that stores the occurence count of unique tokens.


[View source]
def index #

A hash whose values hold a sequential index of a word as it appears in the #count hash.


[View source]
def min_count : Int32 #

[View source]
def min_count=(min_count : Int32) #

[View source]
def pairs #

Iterates over the tokens and constructs Glove::TokenPairs where neighbors hold the adjacent (context) words. The number of neighbors is controlled by #window (on each side).


[View source]
def token_neighbors(word, index) #

Construct an array of neighbors to the given word and its index in the #tokens array.


[View source]
def tokens : Array(String) #

[View source]
def tokens=(tokens : Array(String)) #

[View source]
def window : Int32 #

[View source]
def window=(window : Int32) #

[View source]