GPT-4, the latest version of OpenAI’s primary large language model, exhibits human-level performance on several professional and academic benchmarks. It performs better than its predecessor, GPT-3.5, on a simulated bar exam, for example.
It also shows an array of capabilities that previous versions lacked, including the ability to quickly come up with the proper income tax deduction after being fed reams of tax code. The technology could have a dramatic impact on tech companies and consumers, experts say.
Training
GPT-4, a large multimodal model that accepts text and image inputs and outputs text responses, exhibits human-level performance on various professional and academic benchmarks. For example, in a simulated bar exam it scored in the top 10% of test takers — a significant improvement over its predecessor GPT-3.5.
It can also be used for many other language tasks, including content creation and scientific research. OpenAI says GPT-4 is “more creative and collaborative than ever” and solves difficult problems with greater accuracy, compared to its predecessor.
As for training, GPT-4 was trained using a large and diverse corpus of data, ranging from correct and incorrect solutions to math problems, self-contradictory statements and a variety of ideologies and ideas. It was also tested against adversarial prompts, which can result in the model jumping its guardrails and producing answers that aren’t aligned with the user’s intent.
Inputs
GPT-4 is a large multimodal language model that accepts both text and image inputs and outputs natural language. It’s able to interpret documents with photographs, diagrams, and screenshots as well as text-only inputs.
It’s also more creative, reliable, and generally able to handle more nuanced instructions than its predecessor (GPT-3.5). OpenAI reports that GPT-4 outperformed the former model on a variety of professional and academic benchmarks, including passing a simulated bar exam with an average score around the top 10% of test takers.
GPT-4 has also made progress on external benchmarks, such as the TruthfulQA challenge, which tests a model’s ability to separate fact from an adversarially selected set of incorrect statements. Despite this, it still has limitations such as social biases and hallucinations that need to be addressed.
Outputs
GPT-4 can accept a prompt of text and images, which it can use to generate captions, classifications, analyses, and natural language outputs. It can do this across a range of domains, including documents with text and photographs, diagrams, or screenshots.
The model has also been improved with steerability, or the ability to change its behavior according to user requests. This can help you get the AI to write in a different style or tone or voice, for example.
Despite its capabilities, OpenAI says the GPT-4 model still has some limitations and needs to be used with care. Specifically, it can “hallucinate” facts and make reasoning errors, so it is not yet fully reliable.
OpenAI is working to mitigate these risks and make GPT-4 more aligned. It has engaged with over 50 experts and collected data to improve the model’s ability to refuse dangerous requests. It’s not 100% safe, but it’s much less likely to give inappropriate content than its predecessor and more likely to follow policies that protect sensitive topics like medical advice or self-harm.
Conclusions
OpenAI says its new GPT-4 multimodal model can perform better than its predecessors at a range of professional and academic benchmarks. It’s capable of achieving human-level performance on tasks that require reasoning and problem-solving, including a simulation of a lawyer’s bar exam, SAT reading and math exams, and a variety of standardized tests designed for humans.
Its performance on these tasks is impressive, but it’s not without its limitations. According to the report, GPT-4 still possesses a tendency to hallucinate from time to time and can produce factually incorrect answers.
Despite its shortcomings, GPT-4 is a valuable tool for many organizations that need to understand large amounts of content and text, as well as analyze it. It can also be used for a wide range of applications, including information retrieval, chatbots and social media platforms.