golem15/sm-quizzes-plugin
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b00a9077d503f7b6aa0da460283f1e42b59e6468
sm-quizzes-plugin/README.md
Jakub Zych 76546d5d50 docs: describe open questions, graders and the review queue 
- open type, grading states and student labels, provisional scores
- grader contract (HasGraders, Input with the untrusted answer, Result,
  retries, the reserved manual code, the grading queue)
- the three events, the student-tool rules, review routes and tools,
  the Do oceny screen and the import fields; plugin.yaml regenerated
2026-11-09 18:40:22 +01:00

481 lines
43 KiB
Markdown
Raw Blame History

# golem15.quizzes
A reusable quiz plugin for SummerCMS applications: quizzes with single-choice, multiple-choice, self-check (flashcard) and open questions answered in free text, a learning and a test mode that withholds solutions until an attempt is finished, scored attempts for signed-in users, open answers graded by the owner or by a grader another plugin provides, with an owner review queue, an admin backend, a JSON importer and owner management over a token API and MCP tools.
```go
import _ "git.golem15.com/golem15/sm-quizzes-plugin"
```
Module `git.golem15.com/golem15/sm-quizzes-plugin`, package `quizzes`, plugin ID `golem15.quizzes`.
## Overview
The plugin is a compiled SummerCMS plugin. Its `init` registers it with the framework, so an application loads it by listing it in `summer.yaml`; `summer build` then emits the blank import above into the application's `plugins.gen.go`. It requires `golem15.user`: the API runs behind that plugin's `jwt.auth` middleware, and attempts reference its `users` table.
The plugin knows nothing about the application around it. A quiz links to a record of another plugin through an optional free-form `context` string, such as `acme.courses:course:12`; clients list the quizzes of a context with `GET /quizzes?context=...`.
It owns five tables:
| Table | Holds |
|-------|-------|
| `golem15_quizzes_quizzes` | Quizzes: title, unique slug, description, context, published flag, time limit, passing score, question shuffling, round size, and the test settings `mode`, `navigation` and `max_attempts`, and the `grader` of open answers. |
| `golem15_quizzes_questions` | Questions of a quiz: text, type, category, model answer, explanation, sort order and points, and for open questions the owner-only `rubric` and `max_length`. |
| `golem15_quizzes_answers` | Answers of a single or multiple question, with `is_correct` and an explanation. |
| `golem15_quizzes_attempts` | One user's run through a quiz, or through one category of it, with the score once finished. |
| `golem15_quizzes_attempt_answers` | The user's answer to one question of an attempt; one row per attempt and question. An open answer adds its text, its grading state, the grader's result and the owner's review. |
Deleting a quiz deletes its questions, answers and attempts. Deleting a question deletes its answers and the attempt answers given to it. Deleting a user deletes their attempts.
## Features
- **Four question types.** `single` (one correct answer), `multiple` (one or more correct answers), `self_check`, a flashcard: the client shows the question, the user reveals `answer_text` and marks themselves as knowing it or not, and `open`, a free-text answer graded later. See [Open questions and graders](#open-questions-and-graders).
- **Grading and review.** Open answers are graded by the owner or by a grader another plugin registers, in the background; the owner reviews them in the admin "Do oceny" screen or over MCP, and every review recalculates the attempt.
- **Categories.** A question may carry a category. `GET /quizzes/{slug}?category=` returns one category, and an attempt can be limited to one category.
- **Rounds and shuffling.** `round_size` tells the client to present the questions in rounds of N; `shuffle_questions` shuffles the question order on every `GET /quizzes/{slug}`.
- **Attempts.** A user starts an attempt, answers questions in any order (answering again overwrites the earlier answer and returns fresh feedback) and finishes it. Answers are refused after the time limit; finishing is still allowed and counts the answers given in time.
- **Scoring.** A pure function in `classes` (`classes.Score`, `classes.Totals`) with optional partial credit for multiple questions and an optional penalty for wrong answers. See [Configuration](#configuration).
- **No solution leaks.** The quiz payload never contains `is_correct` or explanations, and `answer_text` only for `self_check` flashcards, which have nothing to hide. Feedback reveals the solution of the question just answered; a result reveals the solutions of answered questions, and of every question once the attempt is finished.
- **Test mode.** A quiz set to `test` withholds every solution until the attempt is finished, and the server enforces the attempt limit, the time limit and one-way navigation. See [Test mode](#test-mode).
- **Admin backend.** Quizzes, Questions (filtered by quiz, category and type), Answers (filtered by quiz) and the review screen "Do oceny" (open answers, filtered by quiz, status and student) under the `golem15.quizzes.access_quizzes` permission, and a read-only Attempts list under `golem15.quizzes.access_attempts`. Polish and English labels.
- **Importer.** `quizzes:import` loads a quiz from a JSON file, idempotent by slug. See [CLI commands](#cli-commands).
## Test mode
A quiz has a `mode`: `learning` (the default; every existing quiz) or `test`. Learning quizzes behave exactly as before test mode existed. Two more settings only apply to tests: `navigation` (`free`, the default, or `one_way`) and `max_attempts` (`null` for unlimited).
- **Withholding.** While a test attempt is open, no student response carries a solution: the answer feedback has `withheld: true` and `null` for `is_correct`, `points`, `correct_answer_ids`, `explanation` and `answer_text`; the result has `solutions_withheld: true` and `null` for `is_correct`, `points_earned`, `correct_answer_ids`, `explanation` and `answer_text` on every question, answered or not, and no answer carries `is_correct` or `explanation`. The student's own input (`answered`, `answer_ids`, `known`) stays. `GET /quizzes/{slug}` of a test lists no questions (`questions: []`, `question_count` counts them), so a timed test cannot be read before its clock starts; the test player reads the questions from `GET /attempts/{id}`.
- **Full review after finish.** Once `completed_at` is set (finish, or a lazy close) the result reveals everything, exactly as for a finished learning attempt, with `passed` against `passing_score`.
- **Attempt limit.** `max_attempts` counts every attempt of the user on the quiz, open or finished, in any mode. A start past the limit is a 409 with `code: attempt_limit_reached`. Learning quizzes have no limit.
- **Resume.** Starting a test while an open, unexpired attempt exists returns that attempt (200, `resumed: true`) instead of creating one, so a page reload does not burn an attempt. Starts are serialised per quiz and user with a Postgres advisory lock (`pg_advisory_xact_lock`), so parallel starts cannot exceed the limit.
- **Time limit and grace.** An answer to a test is refused (409, `code: time_limit_exceeded`) after `started_at + time_limit_seconds` plus a grace for answers still in flight, `golem15.quizzes.test.grace_seconds` (default 5). Learning quizzes have no grace. Finishing stays allowed after the deadline; unanswered questions score 0. `seconds_left` on every attempt row is counted by the server.
- **Lazy close.** A test attempt past its deadline and grace is closed on the next start of that quiz or the next `GET /attempts/{id}`: `completed_at` is set to its deadline and it is scored with the answers given in time, as a finish would. No job queue is involved.
- **Whole quiz only.** A test always covers every question: starting one with a `category` is a 422 on `category` (`category_not_allowed`), and `GET /quizzes/{slug}` of a test ignores `?category=`.
- **Stable order.** With `shuffle_questions`, a test attempt's questions come in an order seeded by the attempt, the same on every `GET /attempts/{id}` and on finish.
- **One-way lock.** With `navigation: one_way` an answer is final: a second answer to the same question is a 409 with `code: answer_locked` and the stored answer is unchanged. With `free` an answer is overwritten until the attempt is finished.
## Open questions and graders
An `open` question takes a free-text answer. `answer_text` is its model answer and `rubric` holds the owner's grading criteria; both are owner-only and never reach a student before the attempt allows it (the rubric never does). `max_length` caps the answer in characters: empty means 2000, at most 20000. An open question has no answers, and `rubric` and `max_length` apply to open questions only.
**Grading states.** Every open answer has a `grading_status`; the student apps show it with these labels:
| Status | Meaning | Student label (pl) |
|--------|---------|--------------------|
| `pending` | Waiting for the grader or for the owner. | "czeka na ocenę" |
| `graded` | A grader scored it; the owner has not confirmed the score. | "do weryfikacji" |
| `failed` | The grader could not score it; the owner grades it. | "czeka na ocenę" |
| `reviewed` | The owner confirmed or set the score. Final. | "zweryfikowano" |
**When grading runs.** In a learning attempt an answer is graded as soon as it is saved; answering again with a different text starts over, the same text changes nothing. In a test the open answers are graded when the attempt closes (finish, lazy close or the stats route), never while it is open.
**Provisional score.** An attempt's score counts each open answer's current points: 0 while pending or failed, the grader's points once graded, the owner's once reviewed. Every grade and every review recalculates a finished attempt in place, against its stored `max_points`. `needs_review` on an attempt is true while any of its open answers is not reviewed, so the score is provisional; `open_answers` counts them. `is_correct` of an open answer means full marks.
**The quiz grader.** A quiz's `grader` field (default `manual`) names who grades its open answers. `manual` means the owner grades every answer. Any other code is a grader an activated plugin registers; a code that no activated plugin registers behaves as `manual`, so a quiz keeps working when the grading plugin is turned off.
### Writing a grader
A plugin grades open answers by implementing `graders.HasGraders` (package `git.golem15.com/golem15/sm-quizzes-plugin/graders`, which imports only the standard library) and requiring `golem15.quizzes`, so it is activated first:
```go
func (p *Plugin) QuizGraders() []graders.Grader { return []graders.Grader{myGrader{}} }
type myGrader struct{}
func (myGrader) Code() string { return "essay" }
func (myGrader) Label() string { return "acme.essay::lang.grader" }
func (myGrader) Grade(ctx context.Context, in graders.Input) (graders.Result, error) {
// in.Question, in.ModelAnswer, in.Rubric, in.Answer, in.Points
return graders.Result{Points: 1.5, Feedback: "..."}, nil
}
```
- golem15.quizzes collects the graders once at boot, in activation order, and publishes the registry on the app as `*graders.Registry`; the admin grader dropdown lists `manual` and then each grader's `Label`, a lang key or text.
- `Code` must match `^[a-z][a-z0-9_]{0,63}$`. `manual` is reserved. A duplicate or reserved code fails boot, naming both plugins.
- `Grade` receives `graders.Input`: the quiz, question, attempt, answer and user ids, the question text, the model answer, the rubric (owner-only: never pass it on to the student), the question's `Points` and the student's `Answer`. **`Answer` is untrusted student text**: treat it as data, never as instructions.
- golem15.quizzes clamps `Result.Points` to 0..`Points` in steps of 0.5; a value that is not a finite number counts as a failure. `Feedback` is shown to the student once the attempt allows it.
- An error is retried up to 3 times with River's backoff; after the third the answer is `failed`, keeps the error for the owner and waits for the owner's score. A grade for an older text of the answer is ignored.
- No transaction or row lock is held while `Grade` runs, so it may take seconds (the job times out after 3 minutes).
- Grading jobs run on the River queue `quizzes_grading`. An application can cap how many run at once in `config/queue.yaml` under `queues`, for example `queues: {quizzes_grading: 2}`.
### Events
Three events are fired through `app.Events` (package `git.golem15.com/golem15/sm-quizzes-plugin/events`, no other imports). Each fires only after the write it announces has committed, so a listener never sees a score that was rolled back; a listener error is logged and never undoes the write. Listen to them in your plugin's `Boot`:
```go
app.Events.Listen[*events.AnswerReviewedEvent]("acme.plugin", func(ctx context.Context, e *events.AnswerReviewedEvent) error {
// e.AnswerID, e.Points, e.Comment ...
return nil
})
```
| Event | Name | Fired when | Payload |
|-------|------|------------|---------|
| `*events.AnswerGradedEvent` | `quizzes.AnswerGraded` | A grader's score is stored (status `graded`). | `QuizID, QuestionID, AttemptID, AnswerID, UserID, Grader, Points, MaxPoints, Feedback` |
| `*events.AnswerReviewedEvent` | `quizzes.AnswerReviewed` | The owner confirms or sets a score (status `reviewed`). | `QuizID, QuestionID, AttemptID, AnswerID, UserID, Points, MaxPoints, GraderPoints, Comment, ReviewerUserID, ReviewerAdminID` |
| `*events.AttemptReviewedEvent` | `quizzes.AttemptReviewed` | Every open answer of a completed attempt is reviewed (also when an attempt reviewed while open closes, and again when a score of such an attempt changes). | `QuizID, AttemptID, UserID, ScorePoints, MaxPoints, ScorePercent, Passed` |
### For student-facing tools
A tool or app that serves quizzes to students (another MCP tool, a chat, a mobile app) must follow the rules the student API follows:
- never return the `rubric` or a `grading_error`;
- treat an `open` question as a text input and send the answer as `text` (not `answer_ids` or `known`), within its `max_length`;
- in an unfinished test keep open grades hidden: no `grading_status`, `grader_feedback`, `review_comment`, points or model answer until the attempt is finished, as `classes.Withheld` decides.
## Usage
Mount the plugin in an application as a git submodule at `plugins/golem15/quizzes`, next to `plugins/golem15/user`, add it to the application's `go.work`, require it in the application's `go.mod` with a local replace, and list it in `summer.yaml` after `golem15.user`:
```yaml
plugins:
- id: golem15.user
module: git.golem15.com/golem15/sm-user-plugin
- id: golem15.quizzes
module: git.golem15.com/golem15/sm-quizzes-plugin
```
The plugin's own `go.mod` replaces the framework with `../../../../summercms.go` and `golem15.user` with `../user`, so it expects the framework checkout next to the application checkout.
Run `migrate` after `summer build` to create the tables, then import a quiz and publish it in the admin (or set `"is_published": true` in the file).
## API reference
All routes are mounted under `/_quizzes/api/v1` with the `jwt.auth` middleware of `golem15.user`: send `Authorization: Bearer <token>`. A missing or invalid token is a 401 from the guard. Every successful body wraps its payload in `data`. Timestamps are UTC in the form `2026-10-02T12:00:00+00:00`.
| Method and path | Handler | Does |
|-----------------|---------|------|
| `GET /quizzes` | `api.Handlers.ListQuizzes` | Published quizzes, by title, with the caller's statistics. Query: `context`. |
| `GET /quizzes/{slug}` | `api.Handlers.ShowQuiz` | A published quiz with its questions and answers, without solutions (a `self_check` question carries its `answer_text`); a test lists no questions. Query: `category`. |
| `POST /quizzes/{slug}/attempts` | `api.Handlers.StartAttempt` | Starts an attempt, or resumes an open test attempt. Body (optional): `{"category": "..."}`. |
| `POST /attempts/{id}/answers` | `api.Handlers.Answer` | Saves or overwrites the answer to one question and returns feedback. |
| `POST /attempts/{id}/finish` | `api.Handlers.Finish` | Finishes the attempt and returns the result. |
| `GET /attempts/{id}` | `api.Handlers.ShowAttempt` | The result with per-question details. |
| `GET /attempts` | `api.Handlers.ListAttempts` | The caller's attempts, newest first. Query: `quiz` (slug), `page`, `per_page` (default 20, at most 100). |
`{slug}` matches `[a-z0-9-]+` and `{id}` matches `[0-9]+`; anything else is a 404.
### Errors
| Status | Body | When |
|--------|------|------|
| 401 | `{"error": true, "message": "Token not provided"}` | No or invalid token (from the guard). |
| 404 | `{"error": "Quiz not found"}` | Unknown or unpublished slug. |
| 404 | `{"error": "Attempt not found"}` | Unknown attempt, or another user's attempt. |
| 409 | `{"error": "The attempt is already completed.", "code": "attempt_completed"}` | Answering or finishing a finished attempt. |
| 409 | `{"error": "The time limit of this attempt has been exceeded.", "code": "time_limit_exceeded"}` | Answering after `started_at + time_limit_seconds` (plus the grace of a test). |
| 409 | `{"error": "All attempts at this test have been used.", "code": "attempt_limit_reached"}` | Starting a test after `max_attempts` attempts. |
| 409 | `{"error": "An answer in this test cannot be changed.", "code": "answer_locked"}` | Answering a question of a `one_way` test again. |
| 413 | `{"error": "Payload too large"}` | A body over the route's limit. |
| 422 | `{"error": "<first message>", "errors": {"<field>": ["<message>"]}}` | Invalid input, the shape `golem15.user` uses. Keys follow the rule order. Starting a test with a category is a 422 on `category`: "A test covers the whole quiz; a category cannot be chosen." |
Every 409 body carries `code`, a stable key clients can branch on; the `error` message is for people. The messages are translated into the request locale (`golem15.quizzes::lang.api.*`, Polish and English).
### GET /quizzes
```json
{"data": [{
"id": 1,
"slug": "security-basics",
"title": "Security basics",
"description": "Flashcards and checks.",
"context": "acme.courses:course:12",
"question_count": 54,
"time_limit_seconds": null,
"passing_score": 50,
"round_size": 5,
"shuffle_questions": false,
"best_score_percent": 83.33,
"attempts_count": 2,
"mode": "test",
"navigation": "free",
"max_attempts": 3,
"attempts_left": 1,
"question_types": {"single": 20, "multiple": 10, "self_check": 24, "open": 0}
}]}
```
`attempts_count` counts the caller's finished attempts and `best_score_percent` is their best finished result (`null` before the first one). `question_types` counts the questions per type, with all four keys always present. `attempts_left` is how many attempts the caller may still start at a test with a limit (every attempt counts, open or finished); it is `null` for a learning quiz or a test without a limit.
### GET /quizzes/{slug}
```json
{"data": {
"id": 1, "slug": "security-basics", "title": "Security basics", "description": null,
"context": "acme.courses:course:12", "time_limit_seconds": null, "passing_score": 50,
"round_size": 5, "shuffle_questions": false,
"question_count": 2,
"categories": ["Documents", "Theory"],
"mode": "learning", "navigation": "free", "max_attempts": null, "attempts_left": null,
"questions": [
{"id": 10, "text": "What is societal resilience?", "type": "self_check", "category": "Theory",
"points": 1, "sort_order": 1, "answer_text": "The ability of a society to withstand a crisis.", "answers": []},
{"id": 11, "text": "Which is a strategy document?", "type": "single", "category": "Documents",
"points": 1, "sort_order": 2, "answers": [{"id": 30, "text": "National Security Strategy"}, {"id": 31, "text": "Highway Code"}]}
]
}}
```
`categories` lists every category of the quiz; with `?category=` the questions are only that category's and `question_count` counts them. An `open` question carries `max_length` (its effective limit, 2000 when unset) and no answers; never its rubric or model answer. Questions come in `sort_order`, or shuffled when `shuffle_questions` is true. Answers come in their sort order. For a test, `questions` is empty, `question_count` counts the whole quiz, `?category=` is ignored and `attempts_left` is set as in the list.
### POST /quizzes/{slug}/attempts
Body, optional: `{"category": "Theory"}` limits the attempt to one category; a category without questions is a 422 on `category`, and a test refuses any category. Answers status 201 with a new attempt, or 200 when an open test attempt is resumed:
```json
{"data": {
"id": 7, "quiz_id": 1, "quiz_slug": "security-basics", "quiz_title": "Security basics",
"category": null,
"started_at": "2026-10-02T12:00:00+00:00",
"expires_at": null,
"completed_at": null,
"answered_count": 0,
"score_points": 0, "max_points": 54, "score_percent": null,
"passing_score": 50, "passed": null,
"quiz_mode": "learning", "quiz_navigation": "free", "seconds_left": null,
"resumed": false, "attempts_left": null
}}
```
`expires_at` is `started_at + time_limit_seconds`, or `null` without a limit. `seconds_left` is the whole seconds until `expires_at`, rounded up and never below 0, counted by the server; it is `null` without a limit and once the attempt is finished. `quiz_mode` and `quiz_navigation` tell the player which flow to run. `max_points` sums the points of the questions the attempt covers. This attempt shape, without `resumed` and `attempts_left`, is also each row of `GET /attempts` and the head of a result. `resumed` is true when an open test attempt was returned; `attempts_left` counts the attempts left after this one (`null` for a learning quiz or without a limit).
### POST /attempts/{id}/answers
Body:
| Field | Rules |
|-------|-------|
| `question_id` | Required integer; the question must belong to the attempt's quiz (and category). |
| `answer_ids` | Array of answer IDs. Required for `single` (exactly one) and `multiple`; every ID must belong to the question. |
| `known` | Boolean. Required for `self_check`. |
| `text` | String. Required for `open`: not blank (422 `text_required`), at most the question's `max_length` characters (422 `text_too_long`). NUL bytes are dropped. |
Checks run in this order: the body (422), the attempt belongs to the caller (404), it is not finished (409), it is within the time limit plus the grace of a test (409), the question belongs to it (422 on `question_id`), a `one_way` test has no answer to it yet (409), the answer fits the question (422 on `answer_ids` or `known`). Answering a question again overwrites the stored answer. Answers status 200:
```json
{"data": {
"question_id": 11,
"withheld": false,
"is_correct": false,
"points": 0,
"correct_answer_ids": [30],
"explanation": "A strategy sets goals and the ways to reach them.",
"answer_text": null,
"answers": [
{"id": 30, "text": "National Security Strategy", "is_correct": true, "explanation": null},
{"id": 31, "text": "Highway Code", "is_correct": false, "explanation": "A traffic law."}
]
}}
```
For an `open` question the feedback adds `text` (the stored answer), `grading_status`, `grader_feedback` and `review_comment`; `is_correct` and `points` stay `null` until the answer is graded or reviewed. `points` can be negative when `scoring.penalty_wrong` is set. For a `self_check` question `correct_answer_ids` and `answers` are empty and `answer_text` holds the model answer. For an open test attempt `withheld` is true and `is_correct`, `points`, `correct_answer_ids`, `explanation` and `answer_text` are `null`; the answers carry only `id` and `text` (`is_correct` and `explanation` `null`).
### POST /attempts/{id}/finish and GET /attempts/{id}
Finishing sums the stored points, floors the total at 0, sets `score_percent = score_points / max_points * 100` (two decimals) and `completed_at`. Finishing a finished attempt is a 409. Both routes answer the attempt shape plus `questions`:
```json
{"data": {
"id": 7, "quiz_id": 1, "quiz_slug": "security-basics", "quiz_title": "Security basics", "category": null,
"started_at": "2026-10-02T12:00:00+00:00", "expires_at": null, "completed_at": "2026-10-02T12:09:41+00:00",
"answered_count": 2, "score_points": 1, "max_points": 2, "score_percent": 50,
"passing_score": 50, "passed": true,
"quiz_mode": "learning", "quiz_navigation": "free", "seconds_left": null,
"solutions_withheld": false,
"questions": [
{"id": 10, "text": "What is societal resilience?", "type": "self_check", "category": "Theory", "points": 1,
"answered": true, "answer_ids": [], "known": true, "is_correct": true, "points_earned": 1,
"correct_answer_ids": [], "explanation": null, "answer_text": "The ability to withstand a crisis...", "answers": []},
{"id": 11, "text": "Which is a strategy document?", "type": "single", "category": "Documents", "points": 1,
"answered": true, "answer_ids": [31], "known": null, "is_correct": false, "points_earned": 0,
"correct_answer_ids": [30], "explanation": "A strategy sets goals...", "answer_text": null,
"answers": [{"id": 30, "text": "National Security Strategy", "is_correct": true, "explanation": null},
{"id": 31, "text": "Highway Code", "is_correct": false, "explanation": "A traffic law."}]}
]
}}
```
An `open` question's row adds `text_answer` (the student's text; `text` is the question), `max_length`, `grading_status`, `grader_feedback` and `review_comment`, with `is_correct` and `points_earned` `null` until it is graded or reviewed; in an open test attempt every grading key is `null`. Every attempt shape (this one, the start answer and the rows of `GET /attempts`) carries `needs_review` and `open_answers`. `passed` is `score_percent >= passing_score` once the attempt is finished, else `null`. While the attempt is open, an unanswered question has `correct_answer_ids`, `explanation` and `answer_text` set to `null` and `is_correct`/`explanation` of its answers set to `null`; its own `is_correct` is `false` and `points_earned` 0. While a test attempt is open `solutions_withheld` is true and every question, answered or not, has `is_correct`, `points_earned`, `correct_answer_ids`, `explanation` and `answer_text` set to `null`, with no answer `is_correct` or `explanation`; `answered`, `answer_ids` and `known` stay. With `shuffle_questions` a test attempt lists its questions in an order seeded by the attempt, the same on every call; other attempts list them in sort order.
### GET /attempts
```json
{"data": [ /* attempt shapes, newest first */ ],
"meta": {"current_page": 1, "last_page": 1, "per_page": 20, "total": 2}}
```
### Packages
- `classes`: `Score`, `Totals` and `ScoringConfigFrom` (scoring); `Service` (the API's queries and writes); `IsTest`, `Withheld`, `OneWay`, `AnswerDeadline`, `Expired`, `SecondsLeft`, `AttemptsLeft`, `AttemptLockKey` and `TestGraceFrom` (the test-mode policy); `ParseImport`, `Import` and `CheckQuestion` (the importer and its per-question rules); `ListManagedQuizzes`, `ManagedQuiz`, `CreateQuiz`, `UpdateQuiz`, `DeleteQuiz`, `SetPublished`, `CreateQuestion`, `UpdateQuestion`, `MergeQuestion`, `DeleteQuestion`, `ReorderQuestions`, `QuizStats` and `BuildStats` (owner management).
- `controllers/api`: `Handlers`, the HTTP handlers.
- `classes` (open questions): `CheckOpenText`, `GraderFor`, `GradeAnswer`, `ForApp`, and the review service `ReviewQueue`, `ReviewItem`, `Confirm`, `SetScore`, `ConfirmBulk` and `Regrade`, shared by the review routes and the admin.
- `controllers`: the admin controllers `QuizzesController`, `QuestionsController`, `AnswersController`, `ReviewsController` and `AttemptsController`.
- `models`: `Quiz`, `Question`, `Answer`, `Attempt`, `AttemptAnswer`, `Review` (an open answer as the admin review screen reads it) and `Slugify`.
- `graders`: `Grader`, `Input`, `Result`, `HasGraders`, `Registry` and `Manual`, the grader contract.
- `events`: `AnswerGradedEvent`, `AnswerReviewedEvent` and `AttemptReviewedEvent`.
- `jobs`: `Queue` (`quizzes_grading`) and `GradeAnswerArgs`.
- `console`: `ImportCommand`.
## Management API and MCP tools
The owner manages quizzes from an MCP client (or any HTTP client) with a `golem15.user` API token. Every route below runs behind `user.api_token`, the per-user throttle `golem15.quizzes.manage` (120 requests a minute), the token scope named in the table and the `golem15.quizzes.access_quizzes` permission. The plugin declares that permission through `pact.HasFrontendPermissions`, so it shows up in golem15.user's User Groups editor. Grant it to the owner's group only, because the stats show student emails and names. Declaring it grants nothing. There is no owner column: a holder of the permission manages every quiz, as in the admin. Quizzes are addressed by slug.
| Method and path under `/api/v1/quizzes` | Scope | MCP tool | Answer |
|---|---|---|---|
| `GET ""` (`?context=`) | read | `quizzes_list_quizzes` | Every quiz, published or not, ordered by context (none last) and title. |
| `GET /{slug}` | read | `quizzes_show_quiz` | The quiz with its questions and answers, solutions included. |
| `POST ""` | write | `quizzes_create_quiz` | 201. The new quiz starts unpublished. A taken slug is a 409. |
| `PATCH /{slug}` | write | `quizzes_update_quiz` | Only sent fields change. `0` clears `time_limit_seconds`, `max_attempts` and `round_size`, and `""` clears `description` and `context`. It cannot publish or change the slug. |
| `DELETE /{slug}?confirm={slug}` | write | `quizzes_delete_quiz` | `{id, slug, title, questions, attempts}`. A 422 unless `confirm` repeats the slug. |
| `POST /{slug}/publish`, `POST /{slug}/unpublish` | write | `quizzes_publish`, `quizzes_unpublish` | The quiz. Both are idempotent. |
| `GET /{slug}/stats` | read | `quizzes_quiz_stats` | `students` and `questions` (see below). |
| `POST /{slug}/questions` | write | `quizzes_create_question` | 201 with the question. It goes last unless `sort_order` is given, and its answers come inline. |
| `PATCH /{slug}/questions/{id}` | write | `quizzes_update_question` | The question. Only sent fields change. A sent `answers` list keeps and updates the answers with an `id`, creates the ones without and deletes the missing ones. |
| `DELETE /{slug}/questions/{id}` | write | `quizzes_delete_question` | `{id, quiz_slug}`. |
| `PUT /{slug}/questions/order` | write | `quizzes_reorder_questions` | The quiz with its questions. `question_ids` must list every question exactly once. |
| `POST /import?replace=` | write | `quizzes_import_quiz` | The import route (see CLI commands). It has its own `throttle:60,1`. With `replace` it deletes the attempt answers of the old questions, open answers with their grades and reviews. |
The quiz shape is `id, slug, title, description, context, mode, navigation, max_attempts, time_limit_seconds, passing_score, shuffle_questions, round_size, grader, is_published, question_count, attempts_count, created_at, updated_at`. The question shape is `id, text, type, category, answer_text, explanation, points, sort_order, rubric, max_length, answers [{id, text, is_correct, explanation, sort_order}]`; `rubric` and `max_length` are `null` unless set on an open question. These owner shapes are never used by a student route.
`grader` on a quiz create or update must be `manual` or a code an activated plugin registers; anything else is a 422 listing the valid codes. `rubric` and `max_length` on a question create or update follow the importer's rules; `""` clears `rubric` and `0` clears `max_length`.
**Review routes.** The open answers are reviewed under a prefix of their own, with the same token, throttle, scopes and permission:
| Method and path under `/api/v1/quiz-reviews` | Scope | MCP tool | Answer |
|---|---|---|---|
| `GET ""` (`?status=&quiz=&user_id=&attempt_id=&page=&per_page=`) | read | `quizzes_review_list` | Open answers, oldest change first. `status` is `open` (the default: pending, graded and failed), a grading status or `all`. Rows carry a 300-character excerpt with `text_truncated`. |
| `GET /{id}` | read | `quizzes_review_show` | One answer with the whole text, the question with its model answer, rubric and `max_length`, the grader's result or error and the review so far. |
| `POST /{id}/confirm` | write | `quizzes_review_confirm` | The grader's score becomes final. 409 `no_grader_score` for a pending or failed answer; a reviewed answer is returned unchanged. |
| `POST /{id}/score` | write | `quizzes_review_set_score` | `{points, comment}`: 0 up to the question's points in steps of 0.5 (else 422), on any status; the comment (at most 2000 characters) replaces an earlier one, and leaving it out clears it. |
| `POST /confirm` | write | `quizzes_review_confirm_bulk` | `{quiz}` or `{attempt_id}`: confirms every graded answer in scope; answers `{confirmed: n}`. |
| `POST /regrade` | write | `quizzes_review_regrade` | `{answer_id}`, `{attempt_id}` or `{quiz}`: pending, graded and failed answers go back to the grader; answers `{regraded: n}`. 409 `already_reviewed` for one reviewed answer, 409 `grader_manual` when the quiz's grader is manual or not registered. |
Every 409 body is `{"error": true, "message", "code"}`. Every review action recalculates a finished attempt and fires the events above.
Input is checked before the database is read. Invalid JSON, an unknown key or a broken rule is a 422 `{"message"}`. A merged question must pass the importer's rules (`classes.CheckQuestion`), and an answer `id` that belongs to another question is refused. A missing quiz or question is a 404.
**History (D-16).** A question can be edited while its quiz has attempts. Finished attempts keep their stored scores, and their reviews show the current question. Answers are updated by id and never deleted and recreated, so the answer ids stored in past attempts stay valid. Deleting a question cascades to its answers and to the attempt answers given to it, and an import with `replace` does the same for every question. On a quiz with attempts, prefer the per-question tools.
**Stats.** The stats route first closes the quiz's expired open test attempts at their deadline. `students [{user_id, email, name, surname, attempts, best_score_percent, last_score_percent, last_completed_at, passed}]` counts each student's completed whole-quiz attempts. Learning rounds of one category are left out, and `passed` is `best_score_percent >= passing_score`. `questions [{id, text, type, answered, correct, correct_percent, most_chosen_wrong}]` counts the answers of every completed attempt. `correct_percent` is `null` when nobody answered. `most_chosen_wrong` is `{answer_id, text, count}`, or `null` for self_check and open questions and when no wrong answer was chosen. Each question also has `unreviewed`, the open answers of completed attempts still pending, graded or failed, and `average_points`, the mean score of an open question's graded and reviewed answers (`null` for other types and before any grade). The report's top-level `unreviewed` is the sum.
## Configuration
Defaults live in `config/config.yaml` and are merged under `golem15.quizzes.*`.
| Key | Default | Meaning |
|-----|---------|---------|
| `golem15.quizzes.scoring.multiple_partial` | `false` | Partial credit on `multiple` questions: `(correct chosen - wrong chosen) / correct answers` of the points, floored at 0. Without it only the exact set earns points. |
| `golem15.quizzes.scoring.penalty_wrong` | `0` | Fraction (0 to 1) of a question's points subtracted for a wrong answer: a non-empty `single` or `multiple` answer that earns nothing. `self_check` answers marked as unknown and empty answers are never penalised. An attempt's total never goes below 0. |
| `golem15.quizzes.test.grace_seconds` | `5` | Seconds after a test's time limit during which an answer still in flight is accepted, clamped to 0..120. A missing or non-numeric value is 5. Learning quizzes have no grace. |
Scores are rounded to two decimals.
### Admin
| Controller | Path | Permission |
|------------|------|------------|
| Quizzes | `golem15/quizzes/quizzes` | `golem15.quizzes.access_quizzes` |
| Questions | `golem15/quizzes/questions` | `golem15.quizzes.access_quizzes` |
| Answers | `golem15/quizzes/answers` | `golem15.quizzes.access_quizzes` |
| Do oceny (open answers) | `golem15/quizzes/reviews` | `golem15.quizzes.access_quizzes` |
| Attempts (read-only) | `golem15/quizzes/attempts` | `golem15.quizzes.access_attempts` |
The quiz form has three test settings: Mode (Learning / Test), Test navigation (Free / One way) and Attempt limit (empty for none); the last two say in their comments that they apply to tests only. It also has the grader dropdown "Ocenianie pytań otwartych": Manual and every grader an activated plugin registers. The quiz list shows the mode and the grader. The question form offers the type "Pytanie otwarte" with the fields rubric (the comment says only the grader sees it) and max_length.
"Do oceny" lists the open answers only, oldest change first, filtered by quiz, status and student. Opening one shows, read-only, the quiz, the student, the question, the model answer, the rubric, the student's text, the grader with its points, feedback and any error, and the current review. Saving the form with "Punkty (ostateczne)" (0 to the question's points, steps of 0.5) and an optional "Komentarz dla studenta" stores the owner's score; a comment without points is refused. The record action "Zatwierdź ocenę" confirms a graded answer, "Oceń ponownie" sends an answer back to the grader (offered only when the quiz has a registered grader and the answer is not reviewed), and the bulk action "Zatwierdź zaznaczone" confirms the selected graded answers. The administrator is stored as the reviewer (`reviewed_by_admin_id`). Answers cannot be created or deleted there. The screen uses the same review service as the MCP tools.
The navigation item is "Quizzes" ("Quizy" in Polish). The admin's relation manager links records through a pivot table only, so questions and answers have their own lists instead of being edited inside the quiz form; filter them by quiz.
## CLI commands
| Command | Purpose |
|---------|---------|
| `quizzes:import <file.json> [--replace]` | Import one quiz from a JSON file. |
The same import is served over HTTP as `POST /api/v1/quizzes/import` (add `?replace=1` for `--replace`), with the import document as the body. The route runs behind the application-wide API token of `golem15.user` (`user.api_token`, created with `user:token:create`): the token needs the `write` scope and its user the `golem15.quizzes.access_quizzes` permission in one of their user groups. It answers `{"data": {"id", "slug", "created", "questions", "answers"}}`, 422 with a `message` for a document that does not parse, and 409 for an existing slug without `replace`. It needs a `golem15.user` that ships the shared API tokens.
The import runs in one transaction and is idempotent by slug:
- a new slug creates the quiz;
- an existing slug is refused unless `--replace` is given;
- with `--replace` the quiz's fields are updated and all its questions are deleted and recreated. The attempts stay, but the attempt answers given to the old questions are deleted with them, open answers with their grades and reviews.
The file is checked before anything is written, and unknown keys are an error. Format:
```json
{
"title": "Security basics",
"slug": "security-basics",
"description": "Flashcards and checks.",
"context": "acme.courses:course:12",
"is_published": true,
"passing_score": 60,
"round_size": 5,
"time_limit_seconds": 1800,
"shuffle_questions": false,
"grader": "manual",
"questions": [
{"text": "What is societal resilience?", "type": "self_check", "category": "Theory",
"answer_text": "The ability of a society to withstand a crisis and recover."},
{"text": "Which is a strategy document?", "type": "single", "category": "Documents",
"explanation": "A strategy sets goals and the ways to reach them.",
"answers": [
{"text": "National Security Strategy", "is_correct": true},
{"text": "Highway Code", "is_correct": false, "explanation": "A traffic law."}
]},
{"text": "Why did the Cold War end?", "type": "open", "points": 2, "max_length": 1500,
"answer_text": "Economic exhaustion of the USSR, reforms and arms control...",
"rubric": "1 point for internal causes, 1 for international ones."},
{"text": "Which countries are NATO members?", "type": "multiple", "points": 2,
"answers": [
{"text": "Poland", "is_correct": true},
{"text": "Sweden", "is_correct": true},
{"text": "Switzerland", "is_correct": false}
]}
]
}
```
| Field | Required | Notes |
|-------|----------|-------|
| `title` | yes | |
| `slug` | no | Normalised to lowercase ASCII with hyphens; built from the title when missing. |
| `description`, `context` | no | |
| `is_published` | no | Default `false`. |
| `passing_score` | no | 0 to 100, default 50. |
| `round_size`, `time_limit_seconds` | no | At least 1; `null` or missing for none. |
| `shuffle_questions` | no | Default `false`. |
| `mode` | no | `learning` or `test`. Missing keeps the quiz's current mode on `--replace` and `learning` on create. |
| `navigation` | no | `free` or `one_way` (tests only). Missing keeps the current value on `--replace` and `free` on create. |
| `max_attempts` | no | At least 1 (tests only). Missing keeps the current value on `--replace` and no limit on create. |
| `grader` | no | A well-formed grader code (lowercase letters, digits and `_`). Missing keeps the current value on `--replace` and `manual` on create. A code no activated plugin registers grades as `manual`. |
| `questions` | yes | At least one. The file order becomes `sort_order`. |
| `questions[].text`, `questions[].type` | yes | `type` is `single`, `multiple`, `self_check` or `open`. |
| `questions[].category`, `answer_text`, `explanation` | no | `answer_text` is a flashcard's answer or an open question's model answer. |
| `questions[].rubric` | no | Open questions only: the owner's grading criteria, never shown to students. |
| `questions[].max_length` | no | Open questions only: 1 to 20000 characters; missing means 2000. |
| `questions[].points` | no | Default 1, not negative. |
| `questions[].answers` | for `single` and `multiple` | `single`: at least two answers, exactly one correct. `multiple`: at least two, one or more correct. `self_check` and `open`: none. |
| `answers[].text` | yes | |
| `answers[].is_correct`, `answers[].explanation` | no | |
`testdata/quiz.json` is a complete example.
## Dependencies
- The SummerCMS framework, `git.golem15.com/golem15/summercms` (modules `backpack`, `bonfire`, `bouncer`, `cabana`, `lagoon`, `pact`, `party`, `phrasebook`, `surf`, `wire`).
- `golem15.user`, `git.golem15.com/golem15/sm-user-plugin`: the `jwt.auth` middleware and the `users` table.
- River, through the framework's `conga` module, runs the grading jobs. Without a job queue, open answers wait for the owner.
- gormigrate and GORM, as pinned in `go.mod`.
- testcontainers-go and pgx for the Postgres-backed tests.
## Testing
```sh
go test -short ./... # fast run, no Docker
go test ./... # full run, starts Postgres with testcontainers (needs Docker)
```
The unit tests cover scoring, request validation, the handlers without a database, the response shapes, the import parser and the admin schema compilation. The Postgres tests run the whole API flow, time limits, cascades, the admin lists and filters and the import command against a throwaway database per test.
Known framework limitations are marked `TODO: requires SummerCMS change` in the code: admin filter choices read the database through `models.FilterDB` because `pact.FilterOptions` receives no context or database; category filter values travel hex-encoded (`models.EncodeCategoryValue`) because the admin accepts only integer or identifier scope values; the quiz form cannot embed its questions because the relation manager handles pivot relations only; and the admin grader dropdown and review screen reach the grader registry and the review service through the package-level hooks `controllers.GraderOptions` and `controllers.Reviews`, because admin hooks receive no app.
Reference in New Issue View Git Blame Copy Permalink