class LLM::Ollama

Overview

Ollama LLM client with context-aware request support

Defined in:

llm/ollama/ollama.cr

Constant Summary

JSON_MODE = JSON::Any.new("json")

format: "json" — Ollama's plain JSON mode.

TEMPERATURE = 0.3

Endpoint extraction wants the model to read code, not to write prose about it, so both request paths pin a low temperature.

Constructors

Class Method Summary

Instance Method Summary

Constructor Detail

def self.new(url : String, model : String) #

[View source]

Class Method Detail

def self.format_value(format : String) : JSON::Any #

Ollama's format field takes either the literal string "json" or a raw JSON Schema, and uses a schema to constrain decoding. The formats in LLM::* are OpenAI-shaped envelopes ({"type":"json_schema","json_schema":{"schema":{...}}}); handing that envelope straight to Ollama constrained generation to the envelope rather than to the endpoint object we asked for, so the response never matched what the analyzer parses — every endpoint in the request was lost. Unwrap to the inner schema, and fall back to plain JSON mode for anything we don't recognise.


[View source]

Instance Method Detail

def request(prompt : String, format : String = "json") #

Make a simple request without context management


[View source]
def request_with_context(system : String | Nil, user : String, format : String = "json", cache_key : String | Nil = nil) #

Make a request with optional context management for improved efficiency


[View source]