DOCS
LLMs

AI, vector databases, and RAG

RailsFast includes built-in support for AI-powered features, vector search, and Retrieval-Augmented Generation (RAG) to help you build intelligent applications.

Overview

Modern Rails applications can leverage AI for:

  • Semantic search and similarity matching
  • Content recommendations
  • Document question-answering
  • Intelligent data retrieval
  • Contextual assistance

Vector Databases

Vector databases store embeddings (numerical representations of text) that enable semantic search.

Supported Solutions

RailsFast works with:

  • pgvector - PostgreSQL extension (recommended for most projects)
  • Pinecone - Managed vector database
  • Weaviate - Open-source vector search engine
  • Milvus - Scalable vector database

Using pgvector

pgvector is included in your PostgreSQL setup.

Enable the Extension 

# db/migrate/xxx_enable_pgvector.rb
class EnablePgvector < ActiveRecord::Migration[8.0]
  def change
    enable_extension 'vector'
  end
end

Create a Model with Embeddings 

# db/migrate/xxx_add_embeddings_to_documents.rb
class AddEmbeddingsToDocuments < ActiveRecord::Migration[8.0]
  def change
    add_column :documents, :embedding, :vector, limit: 1536
    add_index :documents, :embedding, using: :ivfflat, opclass: :vector_cosine_ops
  end
end
# app/models/document.rb
class Document < ApplicationRecord
  has_neighbors :embedding
end

Generating Embeddings

Using OpenAI

Install the OpenAI gem:

# Gemfile
gem 'ruby-openai'

Generate embeddings:

# app/services/embedding_service.rb
class EmbeddingService
  def self.generate(text)
    client = OpenAI::Client.new(
      access_token: Rails.application.credentials.dig(:openai, :api_key)
    )

    response = client.embeddings(
      parameters: {
        model: "text-embedding-3-small",
        input: text
      }
    )

    response.dig("data", 0, "embedding")
  end
end

Use it in your model:

class Document < ApplicationRecord
  has_neighbors :embedding

  after_save :generate_embedding, if: -> { content_changed? }

  private

  def generate_embedding
    update_column :embedding, EmbeddingService.generate(content)
  end
end

Finding Similar Documents 

# Find documents similar to a query
query = "How do I set up payments?"
query_embedding = EmbeddingService.generate(query)

similar_docs = Document.nearest_neighbors(
  :embedding,
  query_embedding,
  distance: "cosine"
).limit(5)

Building a Search Controller 

# app/controllers/search_controller.rb
class SearchController < ApplicationController
  def show
    @query = params[:q]
    return if @query.blank?

    query_embedding = EmbeddingService.generate(@query)

    @results = Document.nearest_neighbors(
      :embedding,
      query_embedding,
      distance: "cosine"
    ).limit(10)
  end
end

RAG (Retrieval-Augmented Generation)

RAG combines vector search with LLMs to provide contextual answers.

Basic RAG Implementation 

# app/services/rag_service.rb
class RagService
  def self.answer_question(question)
    # 1. Find relevant documents
    query_embedding = EmbeddingService.generate(question)

    relevant_docs = Document.nearest_neighbors(
      :embedding,
      query_embedding,
      distance: "cosine"
    ).limit(3)

    # 2. Build context from documents
    context = relevant_docs.map(&:content).join("\n\n")

    # 3. Generate answer with LLM
    client = OpenAI::Client.new(
      access_token: Rails.application.credentials.dig(:openai, :api_key)
    )

    response = client.chat(
      parameters: {
        model: "gpt-4-turbo-preview",
        messages: [
          {
            role: "system",
            content: "Answer questions based on the provided context."
          },
          {
            role: "user",
            content: "Context:\n#{context}\n\nQuestion: #{question}"
          }
        ]
      }
    )

    response.dig("choices", 0, "message", "content")
  end
end

RAG Controller 

# app/controllers/ai_assistant_controller.rb
class AiAssistantController < ApplicationController
  def ask
    @question = params[:question]
    @answer = RagService.answer_question(@question)

    respond_to do |format|
      format.json { render json: { answer: @answer } }
      format.html
    end
  end
end

Advanced Patterns

Combine traditional search with vector search:

class Document < ApplicationRecord
  has_neighbors :embedding

  def self.hybrid_search(query, limit: 10)
    # Vector search
    query_embedding = EmbeddingService.generate(query)
    vector_results = nearest_neighbors(
      :embedding,
      query_embedding,
      distance: "cosine"
    ).limit(limit * 2)

    # Keyword search
    keyword_results = where("content ILIKE ?", "%#{query}%")
      .limit(limit * 2)

    # Combine and deduplicate
    (vector_results + keyword_results).uniq.take(limit)
  end
end

Caching Embeddings

Cache embeddings to reduce API costs:

class EmbeddingService
  def self.generate(text)
    cache_key = "embedding:#{Digest::SHA256.hexdigest(text)}"

    Rails.cache.fetch(cache_key, expires_in: 30.days) do
      # API call to generate embedding
      client = OpenAI::Client.new(...)
      # ... generate and return embedding
    end
  end
end

Batch Processing

Generate embeddings in background jobs:

class GenerateEmbeddingsJob < ApplicationJob
  queue_as :default

  def perform(document_ids)
    documents = Document.where(id: document_ids, embedding: nil)

    documents.each do |doc|
      doc.update(embedding: EmbeddingService.generate(doc.content))
    end
  end
end

# Usage
GenerateEmbeddingsJob.perform_later(Document.pluck(:id))

Configuration

Add your AI service credentials:

EDITOR="cursor --wait" bin/rails credentials:edit
openai:
  api_key: sk-...

# Or for other services
anthropic:
  api_key: sk-ant-...

pinecone:
  api_key: ...
  environment: us-east-1-aws

Performance Tips

  1. Index your embeddings: Use appropriate index types (IVFFlat, HNSW)
  2. Batch embed operations: Generate embeddings in batches for better throughput
  3. Cache frequently used embeddings: Reduce API costs
  4. Use smaller models when possible: text-embedding-3-small is faster and cheaper
  5. Implement retry logic: Handle API failures gracefully

Cost Optimization

  • Cache embeddings aggressively
  • Use background jobs for non-urgent embedding generation
  • Choose appropriate embedding models (smaller = cheaper)
  • Implement rate limiting on user queries
  • Monitor usage and set budgets

Example Use Cases 

# Find relevant docs for a user question
question = "How do I configure email?"
answer = RagService.answer_question(question)

Content Recommendations 

# Find similar articles
article = Article.find(params[:id])
similar = Article.nearest_neighbors(:embedding, article.embedding).limit(5)

Intelligent Support Bot 

# Answer customer questions
customer_query = "Do you offer refunds?"
support_answer = RagService.answer_question(customer_query)

Resources

See Also