Using other AI model Ollama

[AlazaziAmr]

While using Ollama (with llama) the outputs/responses are not good and accurate as openai, is there a special code implementation in ai_assistants.py file to use Ollama(with llama)?

[fjsj]

Hi @AlazaziAmr I believe that's the model fault, not anything specific caused by django-ai-assistant.
But you can double-check what's the input being passed to the model by using Langsmith: https://docs.smith.langchain.com/
All you have to do is set the env vars:

LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=<your-api-key>

If you're using the example projects, set those at your .env file.

[AlazaziAmr]

Hi @fjsj, everything is set correctly, but the Ollama(with llama) can't find the specific function to do the work as the OpenAI.
Maybe it's as you said "the model fault".

[fjsj]

the specific ollama model you're using is really fine-tuned to perform function calling? @AlazaziAmr

[AlazaziAmr]

As mentioned in their website it supports tool calling. @fjsj

same example showen in the above picture performed with gpt-4o and work perfectly.

[fjsj]

Did you double-check with Langsmith or with logs the request made to the model? Are the function tools being passed?

[AlazaziAmr]

No the function tools are not passed

[fjsj]

What ChatModel are you using? Should be one that supports "Tool calling" from this table: https://python.langchain.com/docs/integrations/chat/#featured-providers

Is it really ChatOllama?

[AlazaziAmr]

Yes it's ChatOllama, and I think the function tools are passed acording to the this picture

but maybe the model does not know it !?

[fjsj]

Yes, it seems right. Looks like it's a model problem.

[AlazaziAmr]

Yes, I'll try to dig deeper into it, and find the problem, and let you know about the updates.

Thank you so much for your time, appreciate it.