A couple years ago at the start of this site, I created a quizzes section to learn Livewire, which quickly became popular.

The first 250 questions or so were hand made by me from the documentation, and it was tedious to say the least.

I stopped working on that area for a long time, and kept telling myself every week I'd go back to it but I never found the time, and I knew it was getting stale.

A couple weeks ago a lightbulb went off, and I thought what if I just use OpenAI to generate the rest of the questions.

So that's what I did.

This is what the backend of the quizzes section looks like:

I can manage topics (right now just we just have Laravel filled out)

I can manage the categories in each topic:

And I can manage the questions in each category:

You may have noticed the button in the upper right of the last screenshot called "AI Generation", let's see what that does when we click it:

It opens up a modal that asks for a link to any documentation, so lets put in the documentation link for the Artisan Console:

I'll explain the OpenAI call in a second, but this is what happens after you click submit:

It opens up another modal and gives me 10 generated questions from that documentation, each with 4 options, and it tells me which are correct. It generates single and multiselect questions and even gives an explanation and link to the exact part of the documentation.

I go through each quick and uncheck the "Use" checkbox at the top if I don't want it, it will also give me a red banner that says "Duplicate" if the direct documentation link already exists in my database. So let's see how it works.

When the submit button is clicked, we make a call out to OpenAI:

 1public function generateTestQuestions(string $language, string $documentationLink): array
 2{
 3    $response = $this->client->post('https://api.openai.com/v1/chat/completions', [
 4        'headers' => [
 5            'Authorization' => 'Bearer '.$this->apiKey,
 6            'Content-Type' => 'application/json',
 7        ],
 8        'json' => [
 9            'model' => 'gpt-4o',
10            'messages' => [
11                [
12                    'role' => 'system',
13                    'content' => 'You are a '.$language.' expert and have read the entire documentation.',
14                ],
15                [
16                    'role' => 'user',
17                    'content' => "
18Generate 10 test questions for a topic based on this documentation: $documentationLink. There can be single or multiple select questions. Tell me which option(s) are correct and an explanation of why. If the answer options contain code, please surround them in 3 backticks. Use this format:
19 
20{STARTQUESTION}Which of the following drivers are supported out of the box by Laravel Filesystem? (Choose all that apply){ENDQUESTION}
21{STARTOPTIONS}
22{OPTION}A.{ENDID} Google Cloud Storage{ENDOPTION}
23{OPTION}B.{ENDID} Amazon S3{ENDOPTION}
24{OPTION}C.{ENDID} Dropbox{ENDOPTION}
25{OPTION}D.{ENDID} FTP{ENDOPTION}
26{OPTION}E.{ENDID} Local{ENDOPTION}
27{OPTION}F.{ENDID} Microsoft Azure Blob Storage{ENDOPTION}
28{ENDOPTIONS}
29{CORRECTOPTIONS}B, D, E{ENDCORRECTOPTIONS}
30{EXPLANATION}Laravel's filesystem supports Local, Amazon S3, and FTP drivers out of the box. Support for other drivers can be added via third-party packages.{ENDEXPLANATION}
31{LINK}http://laravel.com/*{ENDLINK}
32{VERSION}11.x{ENDVERSION}
33{TYPE}radio||checkbox{ENDTYPE}
34{ENDBLOCK}
35                    ",
36                ],
37            ],
38        ],
39    ]);
40 
41    $body = $response->getBody();
42    $content = json_decode($body, true);
43 
44    return $this->extractQuestionParts($content['choices'][0]['message']['content']);
45}

So it looks like a lot but it's actually really simple, OpenAI expects a messages array. The first message with the role of system tells OpenAI that it should mold its answers around being an expert in the language or framework we chose.

The second message tells OpenAI exactly what I want, which is to Generate 10 test questions for a topic based on this documentation I then tell it the exact format I want it to respond in to make it easier for me to parse.

This is the method that parses the response:

 1public function extractQuestionParts(string $response): array
 2{
 3    $formatted = [];
 4    $questions = collect(explode('{ENDBLOCK}', $response))
 5        ->filter(fn ($item) => $item !== '' && $item !== "\n")
 6        ->values()
 7        ->toArray();
 8 
 9    foreach ($questions as $index => $question) {
10        $formatted[$index] = [
11            'question' => getStringBetween($question, '{STARTQUESTION}', '{ENDQUESTION}'),
12        ];
13 
14        $correctOptions = explode(', ', getStringBetween($question, '{CORRECTOPTIONS}', '{ENDCORRECTOPTIONS}'));
15        $formatted[$index]['correctOptions'] = $correctOptions;
16 
17        $options = collect(explode("\n", getStringBetween($question, '{STARTOPTIONS}', '{ENDOPTIONS}')))
18            ->filter(fn ($item) => $item !== '')
19            ->values()
20            ->toArray();
21 
22        foreach ($options as $option) {
23            $optionParts = getStringBetween($option, '{OPTION}', '{ENDOPTION}');
24            $optionParts = explode('{ENDID}', $optionParts);
25            $optionLetter = trim(rtrim($optionParts[0], '.'));
26 
27            $formatted[$index]['options'][$optionLetter] = [
28                'option' => $option = trim($optionParts[1] ?? 'Error'),
29                'type' => str_contains($option, '```') ? 'markdown' : 'plain',
30                'correct' => in_array($optionLetter, $correctOptions, true),
31            ];
32        }
33 
34        $formatted[$index]['explanation'] = getStringBetween($question, '{EXPLANATION}', '{ENDEXPLANATION}');
35        $formatted[$index]['version'] = getStringBetween($question, '{VERSION}', '{ENDVERSION}');
36        $formatted[$index]['documentation'] = getStringBetween($question, '{LINK}', '{ENDLINK}');
37        $formatted[$index]['type'] = getStringBetween($question, '{TYPE}', '{ENDTYPE}');
38        $formatted[$index]['use'] = true;
39    }
40 
41    return $formatted;
42}

I then take the formatted array and use Eloquent to save it to the appropriate tables and relationships.

I added about 1,000 quiz questions over the course of a week for about $6 in OpenAI usage.

Hope this helps anyone wanting to play around with AI!