Never call a model in a Rails request
The one absolute rule of Rails AI integration. Everything else is a judgment call.
There is exactly one absolute rule in Rails AI integration: never call a model inside a web request.
Model calls take one to sixty seconds. Your request timeout is thirty. Your users’ patience is three. The math does not work, and it fails in the worst possible way: intermittently, under load, in production.
# No. This will page you at 2am.
def create
reply = LlmClient.chat(prompt: params[:question])
render json: { reply: reply }
end
# Yes. The job owns the slowness.
def create
ticket = Ticket.create!(question: params[:question], status: :pending)
DraftReplyJob.perform_later(ticket.id)
render json: { id: ticket.id, status: "pending" }, status: :accepted
end
The job version buys you everything the request version cannot have:
- Retries with backoff when the provider has a bad minute, and providers have bad minutes weekly.
- A timeout you control, instead of a gateway 504 you do not.
- A place to record tokens, latency, and cost per call.
- Idempotency, so a retry does not send two emails or charge two cards.
The frontend polls or subscribes over Turbo Streams. Users tolerate a visible “thinking” state far better than a spinner that dies at thirty seconds.
If your current AI feature calls the model synchronously and has not hurt you yet, it is because your traffic is low. That is not the same thing as the design being right.
Tomorrow: an AI tool is just a service object. And if this series is describing problems someone on your team is living with right now, the AI Readiness Assessment is two weeks and tells you exactly where the fixes rank.