Function Calling: Ollama vs OpenAI
Function calling is a feature that often gets misunderstood due to its misleading name. In this video, Matt Williams explains that function calling doesn’t mean the model calls functions directly; instead, it formats the output in a way that allows users to call functions reliably. He critiques the naming convention and clarifies the actual process of function calling using OpenAI’s API.
Matt demonstrates the steps involved in OpenAI’s function calling, which includes informing the model about the function schema, parsing the model’s output to JSON, and then calling the function based on this output. He highlights that while the feature is useful, its name doesn’t accurately describe its function.
The video then shifts focus to Ollama, comparing its function calling feature, referred to as ‘format JSON,’ which is more descriptive. Using Python, Matt shows how to set up a payload with a model and an array of messages, send a post request, and print the JSON response. He demonstrates a dynamic example where the country name is taken as a command line argument, and the model returns the capital city’s coordinates in JSON format.
Matt further enhances the example by calculating the distance between Bainbridge Island and the capital city using the ‘haversine’ package. He addresses potential issues with inconsistent key names in the JSON response by providing a schema to the model, similar to OpenAI’s function block. He also shows how to use few-shot prompting to improve the model’s adherence to the schema.
Despite these enhancements, Matt notes that asking a model for latitude and longitude isn’t always precise and suggests setting the temperature to zero to improve consistency. He concludes that while function calling (or format JSON) is a valuable feature for getting consistent output, the implementation in OpenAI’s API is more complicated compared to Ollama’s approach.
Matt encourages viewers to share their thoughts on function calling and suggest features they’d like to see covered in future videos. He mentions that viewer comments have significantly expanded his list of video ideas, indicating an active and engaged audience. The video ends with a thank you to the viewers and an invitation to continue the discussion in the comments.