fix: H1 conversion leaves code alone in markdown that starts with HTML

ConvertContentH1 reads content as markdown on every path and converts it
outside goldmark's code ranges only; a body opening with an HTML block
keeps the h1 tags of its code samples, a raw h1 outside code is converted
as in v0.3.1 (the 33-15 test pin for that case is reversed).

quick-261104-xae, review WR-01.
This commit is contained in:
Jakub Zych
2026-11-05 00:17:53 +01:00
parent 0a62090435
commit a41d537e52
4 changed files with 111 additions and 16 deletions

View File

@@ -178,9 +178,9 @@ The search document holds the default-locale title, excerpt and content; content
Journal decides per site whether its rich text may hold an H1 (2026-10-28, D-50): a page template that shows the post title or the category name as the H1 needs the body to start at H2, and Journal cannot know which template a site uses.
- **The setting.** Settings › Journal, tab Headings and links, switch `content_h1_to_h2` (Convert H1 to H2 in post and category content), on by default. The settings row decides; the config key `golem15.journal.content_h1_to_h2` is used only when no row exists (`classes.EffectiveH1ToH2`). Switching it never rewrites stored content.
- **On.** Every H1 saved in Journal rich text becomes an H2 (`classes.ConvertH1`, on the `golang.org/x/net/html` tokenizer: only the tag name changes, attributes and every other byte are kept), on every write path: an admin save of a post or a category (notice `editor.h1_converted` or `editor.h1_converted_category` on the editor field), `POST` and `PUT /_journal/api/v1/posts` and so MCP (default and translated `content` and `content_html`; the answer gains `notices: [{message, count, locale}]`, `locale` only on a translation's notice), markdown content (its rendered HTML only: a markdown source is never rewritten, so a code sample that shows `<h1>` keeps it; `classes.ConvertContentH1` rewrites `content` only when it is HTML, `classes.LooksLikeHTML`) and `journal:import-posts` (it prints `Converted N H1 headings to H2.`). The bytes of an unfinished tag at the end of a text are kept. The editor offers H2 to H4.
- **On.** Every H1 saved in Journal rich text becomes an H2 (`classes.ConvertH1`, on the `golang.org/x/net/html` tokenizer: only the tag name changes, attributes and every other byte are kept), on every write path: an admin save of a post or a category (notice `editor.h1_converted` or `editor.h1_converted_category` on the editor field), `POST` and `PUT /_journal/api/v1/posts` and so MCP (default and translated `content` and `content_html`; the answer gains `notices: [{message, count, locale}]`, `locale` only on a translation's notice), markdown content (its source is converted outside code only; `classes.ConvertContentH1` reads `content` as markdown, as the renderer does: an `<h1>` in a code block or code span keeps its tag, also when the body starts with an HTML block, and a raw `<h1>` elsewhere becomes `<h2>`, as in the rendered HTML) and `journal:import-posts` (it prints `Converted N H1 headings to H2.`). The bytes of an unfinished tag at the end of a text are kept. The editor offers H2 to H4.
- **Off.** H1 is stored as written, no notice appears, the editor offers H1 to H4 and `journal:fix-headings` refuses.
- **Stored content.** `journal:fix-headings` (a dry run unless `--apply`) and the MCP tool `journal_fix_headings` (`dry_run` default `true`) convert H1 already stored in post content (default and translated; of a markdown post only `content_html`) and in category pillar text, in one transaction, firing a `RecordSaved` per changed post and category; they answer the post, category and heading counts and refuse while the setting is off (exit code 1; `409` from the API).
- **Stored content.** `journal:fix-headings` (a dry run unless `--apply`) and the MCP tool `journal_fix_headings` (`dry_run` default `true`) convert H1 already stored in post content (default and translated; in a markdown source outside code only) and in category pillar text, in one transaction, firing a `RecordSaved` per changed post and category; they answer the post, category and heading counts and refuse while the setting is off (exit code 1; `409` from the API).
- **The API flag.** `GET /_journal/api/v1/posts/{slug}` and every category row of `GET /_journal/api/v1/categories` carry `content_h1_to_h2` (boolean, an added key), so a frontend reads the setting from the Journal and not from its own option.
- **Generic link texts.** The `generic_anchors` setting (one per line; the English and Polish seed lists when empty, `classes.DefaultGenericAnchors`) is the list the editor's outline and link form warn about.
- **Settings API.** `GET` and `PUT /_journal/api/v1/settings` read and write every settings column for `golem15.journal.manage_settings` (`403` without it); `PUT` checks the submitted columns against `models.Settings.Rules` and ignores other keys.
@@ -492,7 +492,7 @@ The request is a `POST` to the URL with `Authorization: Bearer <token>`, `Conten
| `classes.GenericAnchors`, `classes.DefaultGenericAnchors` | The generic link texts: the settings row's list, one per line, lowercased, else the default English and Polish lists. |
| `classes.ConvertH1` | Turns every h1 of HTML into an h2 on the tokenizer, keeping every other byte; returns the converted count. |
| `classes.CountH1` | The number of h1 start tags `ConvertH1` would convert. |
| `classes.ConvertContentH1`, `classes.ConvertTranslationH1` | `ConvertH1` over a post's `content` and `content_html`, and over one locale's translated attributes. |
| `classes.ConvertContentH1`, `classes.ConvertTranslationH1` | `ConvertH1` over a post's `content` (outside markdown code) and `content_html`, and over one locale's translated attributes. |
| `classes.FixStoredHeadings`, `classes.HeadingFix` | The stored-H1 conversion behind `journal:fix-headings` and `POST /fix-headings`: posts in every locale and category pillar text, counted or applied in one transaction with record events. |
| `classes.H1ConversionOffText`, `classes.ErrH1ConversionOff` | The refusal while the H1 setting is off. |
| `classes.SetImageAlt` | Sets the alt of the matching img through the tokenizer; returns the HTML, the match count and the old alt. |

View File

@@ -70,27 +70,55 @@ func containsFold(s, sub string) bool {
// ConvertContentH1 applies ConvertH1 to a post body and its rendered HTML
// (content and content_html) and returns both with the number of headings
// converted: content_html's count, or content's when content_html had none.
// content is rewritten only when it is HTML (LooksLikeHTML, the rich
// editor's body): a markdown body is never rewritten, so a code sample that
// shows an <h1> keeps it, and only its rendered content_html is converted
// (every render path derives content_html from the markdown again and
// converts it again).
// content is read as markdown, as every render path reads it (FormatHTML
// passes raw HTML through, so an HTML body is HTML blocks and is converted
// as before): an <h1> inside a code block or code span keeps its tag
// whatever the body's first character, so a markdown post that opens with
// an HTML block keeps its code samples (33-REVIEW-GAPFIX WR-01), and a raw
// <h1> outside code becomes <h2>, as in the rendered HTML.
func ConvertContentH1(content, contentHTML string) (string, string, int) {
contentHTML, nHTML := ConvertH1(contentHTML)
if !LooksLikeHTML(content) {
return content, contentHTML, nHTML
}
content, nContent := ConvertH1(content)
content, nContent := convertH1OutsideCode(content)
if nHTML > 0 {
return content, contentHTML, nHTML
}
return content, contentHTML, nContent
}
// convertH1OutsideCode applies ConvertH1 to the parts of markdown md
// outside its fenced and indented code blocks and code spans (goldmark's
// AST, the engine of FormatHTML): an h1 shown in a code sample keeps its
// tag. HTML blocks are not code, so an HTML body is converted whole. md
// without an h1 tag comes back unchanged with 0 without being parsed.
func convertH1OutsideCode(md string) (string, int) {
if !containsFold(md, "<h1") && !containsFold(md, "</h1") {
return md, 0
}
var b strings.Builder
b.Grow(len(md))
count := 0
last := 0
apply := func(part string) {
out, n := ConvertH1(part)
count += n
b.WriteString(out)
}
for _, r := range markdownCodeRanges(md) {
apply(md[last:r[0]])
b.WriteString(md[r[0]:r[1]])
last = r[1]
}
apply(md[last:])
if count == 0 && b.String() == md {
return md, 0
}
return b.String(), count
}
// ConvertTranslationH1 applies ConvertContentH1 to the string content and
// content_html of one locale's translated attributes, in place, and returns
// the number of headings converted. A markdown content is never rewritten
// (ConvertContentH1). Other attributes and values that are not strings are
// the number of headings converted. content is rewritten outside code
// only (ConvertContentH1). Other attributes and values that are not strings are
// left alone.
func ConvertTranslationH1(attrs map[string]any) int {
if attrs == nil {

View File

@@ -75,12 +75,41 @@ func TestConvertContentH1(t *testing.T) {
if n != 1 || !strings.Contains(h, "<h2") || strings.Contains(h, "<h1") || !strings.Contains(h, "&lt;h1&gt;Hello&lt;/h1&gt;") || !strings.Contains(h, "<code>&lt;h1&gt;</code>") {
t.Fatalf("rendered HTML = %q n=%d", h, n)
}
// Markdown without rendered HTML: nothing changes, nothing counted.
if c, h, n := ConvertContentH1("plain <h1>x</h1> text", ""); c != "plain <h1>x</h1> text" || h != "" || n != 0 {
// A raw h1 outside code is a heading in the rendered page, so it is
// converted in the markdown source too (as v0.3.1 did), and counted
// when there is no rendered HTML.
if c, h, n := ConvertContentH1("plain <h1>x</h1> text", ""); c != "plain <h2>x</h2> text" || h != "" || n != 1 {
t.Fatalf("markdown without HTML = %q %q %d", c, h, n)
}
}
// TestConvertContentH1HTMLLead pins markdown that starts with an HTML
// block (33-REVIEW-GAPFIX WR-01): the first byte does not make it HTML,
// so the h1 samples in its code keep their tags and only a raw h1 outside
// code is converted.
func TestConvertContentH1HTMLLead(t *testing.T) {
probe := "<p class=\"lead\">Intro</p>\n\n```html\n<h1>Hello</h1>\n```"
rendered, err := FormatHTML(probe)
if err != nil {
t.Fatal(err)
}
for _, html := range []string{rendered, ""} {
c, h, n := ConvertContentH1(probe, html)
if c != probe || h != html || n != 0 {
t.Fatalf("probe with content_html %q = %q %q n=%d", html, c, h, n)
}
}
mixed := "<p class=\"lead\">Intro</p>\n\n<h1>Real</h1>\n\n```html\n<h1>Hello</h1>\n```"
want := "<p class=\"lead\">Intro</p>\n\n<h2>Real</h2>\n\n```html\n<h1>Hello</h1>\n```"
if c, h, n := ConvertContentH1(mixed, ""); c != want || h != "" || n != 1 {
t.Fatalf("raw h1 outside code = %q %q n=%d", c, h, n)
}
span := "<p>x</p>\n\nUse `<h1>` here"
if c, _, n := ConvertContentH1(span, ""); c != span || n != 0 {
t.Fatalf("code span = %q n=%d", c, n)
}
}
// TestConvertTranslationH1 pins a locale's translated attributes,
// changed in place: content and content_html converted, other attributes
// and values that are not strings left alone (a category's pillar text
@@ -102,4 +131,13 @@ func TestConvertTranslationH1(t *testing.T) {
if n := ConvertTranslationH1(tr); n != 0 || tr["content"] != md {
t.Fatalf("a markdown translation was rewritten: %v n=%d", tr, n)
}
probe := "<p class=\"lead\">Intro</p>\n\n```html\n<h1>Hello</h1>\n```"
rendered, err := FormatHTML(probe)
if err != nil {
t.Fatal(err)
}
lead := map[string]any{"content": probe, "content_html": rendered}
if n := ConvertTranslationH1(lead); n != 0 || lead["content"] != probe {
t.Fatalf("markdown with an HTML lead was rewritten: %v n=%d", lead, n)
}
}

View File

@@ -321,6 +321,35 @@ func TestHeadingConversion(t *testing.T) {
}
})
t.Run("markdown that starts with an HTML block keeps its h1 samples", func(t *testing.T) {
setH1ToH2(t, env, true)
md := "<p class=\"lead\">Intro</p>\n\n# Title\n\n```html\n<h1>Hello</h1>\n```"
id := apiPostID(t, env.expect(t, http.StatusCreated, http.MethodPost, posts, token, map[string]any{"title": "HTML lead", "content": md}))
p := storedPost(t, env, id)
if p.Content != md || strings.Contains(p.ContentHTML, "<h1") || !strings.Contains(p.ContentHTML, "&lt;h1&gt;Hello&lt;/h1&gt;") || !strings.Contains(p.ContentHTML, "<h2") {
t.Fatalf("API save: content = %q, content_html = %q", p.Content, p.ContentHTML)
}
dir := t.TempDir()
raw, err := json.Marshal(console.PostFile{Slug: "imported-html-lead", Title: "Imported HTML lead", Content: md})
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "imported-html-lead.json"), raw, 0o644); err != nil {
t.Fatal(err)
}
var buf strings.Builder
if err := console.ImportPosts(env.app).Run(t.Context(), cmdInput{"path": dir}, bonfire.NewOutput(strings.NewReader(""), &buf, &buf)); err != nil {
t.Fatalf("import: %v (%s)", err, buf.String())
}
var imported models.Post
if err := env.gdb.Where("slug = ?", "imported-html-lead").Take(&imported).Error; err != nil {
t.Fatal(err)
}
if imported.Content != md || strings.Contains(imported.ContentHTML, "<h1") {
t.Fatalf("import: content = %q, content_html = %q", imported.Content, imported.ContentHTML)
}
})
t.Run("category pillar text through the admin", func(t *testing.T) {
catToken := env.admin(t, controllers.PermissionAccessCategories)
save := func(t *testing.T, slug string) (*string, []adminNotice) {