On August 7, 2024, OpenAI announced the introduction of a breakthrough feature-structured output in its API, which significantly improved the reliability and application accuracy of the API. This new feature ensures that the output generated by the model fully conforms to the JSON schema provided by the developer, solving the limitation of previous JSON schemas that could not guarantee that the output fully conforms to a specific schema. In addition, OpenAI also announced that the price of the new model has been reduced, with the input price halved and the output price reduced by one-third, which is undoubtedly great news for developers. The editor of Downcodes will explain this technological innovation in detail.
On August 7, 2024, OpenAI today announced the introduction of a breakthrough feature-structured output in its API. This new feature ensures that the output generated by the model fully conforms to the JSON schema provided by the developer, significantly improving the reliability of the API and the accuracy of the application. Unlike the previous JSON schema, the current structured output can not only define the Schema of JSON (that is, what the structure of the JSON output looks like), but also ensure that the output JSON is 100% correct. At the same time, OpenAI announced that the price of the new model has been reduced. The input price is only half of the original price, and the output is also 1/3 cheaper.
Last year, OpenAI introduced JSON Schema as a powerful tool for developers building applications. However, while JSON Schema improves the likelihood of producing valid JSON output, it does not guarantee that the output conforms to a specific schema. The introduction of structured output is precisely to solve this limitation, by constraining the model output to match the preset pattern, ensuring the accuracy and consistency of the output data.
In OpenAI's tracking evaluation of complex JSON patterns, the new model gpt-4o-2024-08-06 achieved an excellent score of 100% in structured output, while the old model gpt-4-0613 scored less than 40% . This significant progress not only demonstrates OpenAI's technological leap, but also brings higher work efficiency and fewer error rates to developers.
OpenAI’s API introduces two forms of structured output:
1. Function call: Structured output can be used by setting it in the function definition. This feature is available for all tools-enabled models, including All models and higher. When structured output is enabled, the model output will match the provided tool definition.
API request settings
API response
2. New option of parameter response_format : Developers can respond to user requests in a structured manner by providing JSON schema json_schema as a new option of parameter.
Security is always OpenAI’s top priority. The new structured output feature respects existing security policies, allowing models to reject unsafe requests when necessary. In addition, OpenAI's Python and Node SDKs have been updated to natively support structured output, simplifying the development process.
Structured output can be used for a wide range of applications, including dynamically generating user interfaces based on user intent, separating final answers from supporting reasoning, and extracting structured data from unstructured data. These use cases demonstrate the practicality and flexibility of structured output in different scenarios.
OpenAI achieves 100% matching between model output and JSON Schema by using restricted decoding technology. This approach ensures that at each step, the model can only select valid tokens by dynamically constraining decoding, thereby reducing the possibility of errors.
OpenAI's structured output function not only provides developers with a more reliable and accurate tool, but also further promotes the advancement of artificial intelligence technology in data processing and application development. With the official launch of this feature, we look forward to seeing more innovative applications being launched.
All in all, OpenAI's structured output function is an important technological advancement that will significantly improve the reliability and efficiency of AI applications and bring a more convenient development experience to developers. The editor of Downcodes believes that this technology will play an increasingly important role in the development of AI applications in the future.