Which GPT Model to choose?

To interact with a GPT model through the OpenAI API, you’ll transmit a request that includes your inputs and your API key. In turn, you will get a response featuring the output from the model. OpenAI’s most recent models, gpt-4 and gpt-3.5-turbo, can be reached via the chat completions API endpoint. At present, it is only the previous, legacy models that are accessible through the completions API endpoint.

Our general advice is to utilize either gpt-4 or gpt-3.5-turbo, though the optimal choice depends on the intricacy of the tasks you’re employing the models for. Gpt-4 typically outperforms across a broad spectrum of evaluations and is particularly proficient at meticulously adhering to complex directives. In comparison, gpt-3.5-turbo tends to follow only a single aspect of a multi-part command. When it comes to fabricating information, a phenomenon referred to as “hallucination”, gpt-4 is less prone to this compared to gpt-3.5-turbo. Additionally, gpt-4‘s context window is larger, accommodating up to 8,192 tokens as opposed to gpt-3.5-turbo‘s 4,096 tokens. However, gpt-3.5-turbo compensates by offering outputs with reduced latency and at a significantly lower cost per token.

To see how many tokens are in a text string without making an API call, use tiktoken Python library.

We suggest trying out various models in the playground to discover which offers the best balance between cost and performance for your needs. A prevalent design pattern involves using several unique query types, each delegated to the model best suited to manage them.

Conclusion

Understanding the optimal ways to engage with GPTs can drastically improve the performance of your applications. It’s essential to acknowledge that GPTs can exhibit specific failure modes, and the strategies to mitigate or rectify these aren’t always straightforward.

The proficiency of working with GPTs, colloquially termed as “prompt engineering”, has evolved over time, surpassing the mere crafting of prompts to designing systems that incorporate model queries as elements. To delve deeper, refer to our guide on GPT best practices which details techniques to enhance model reasoning, minimize the chances of model hallucinations, among other things. Moreover, the OpenAI Cookbook is a trove of valuable resources, including code samples.