398 lines
37 KiB
Markdown
398 lines
37 KiB
Markdown
|
|
# Page rendering docs
|
||
|
|
|
||
|
|
Every page in a project has a rendering doc: one Markdown file that records, section by section, what the server HTML of each template holds and when each section's JavaScript loads. Search engines are certain to read only a page's server HTML, and Googlebot Smartphone crawls the mobile template, so a doc is written from Googlebot's point of view and checked against the real HTML. It is also where the team sees, before a change, which sections ship JavaScript. Default sections for common page types are in [page-type-maps.md](page-type-maps.md), the crawler rules a doc checks against are in [ui-rules.md](ui-rules.md#crawlers), and the way to choose where a section renders is in [rendering-patterns.md](rendering-patterns.md#choosing-where-a-part-loads).
|
||
|
|
|
||
|
|
Read the sections a task needs. To design a page: Where docs live, Vocabulary, Doc format, Index file, and Writing a doc. To write a missing doc: Where docs live, Vocabulary, Doc format, Index file, Producing a doc from code, and Checking the crawler view. To audit a page: Vocabulary, Doc format, Producing a doc from code, and Checking the crawler view. To change or review a page that has a doc: Vocabulary, Doc format, and Keeping docs current, then Checking the crawler view once the code is built. Doc format, Index file, and Example: home page hold `##` lines inside code blocks, so read each of them to the next heading that the Contents line names.
|
||
|
|
|
||
|
|
Contents: Where docs live · Vocabulary · Doc format · Index file · Writing a doc · Producing a doc from code · Checking the crawler view · Keeping docs current · Example: home page
|
||
|
|
|
||
|
|
## Where docs live
|
||
|
|
|
||
|
|
```
|
||
|
|
docs/pages/
|
||
|
|
├── README.md # the index file: one line per page and per layout
|
||
|
|
└── <app>/
|
||
|
|
├── layout-shop.md # the (shop) group's layout: header and footer, documented once
|
||
|
|
├── not-found.md # the app's not-found and error pages
|
||
|
|
├── home.md # /
|
||
|
|
├── product-id.md # /product/[id]
|
||
|
|
└── category-slug.md # /category/[...slug]
|
||
|
|
```
|
||
|
|
|
||
|
|
- `docs/pages/` sits at the monorepo root, beside `docs/decisions/`. Create it with the first doc.
|
||
|
|
- Write one doc per route, that is, per `page.tsx`. Name it from the route: drop route groups, join the segments with `-`, write a dynamic segment as its parameter name, and call `/` `home`. Brackets stay out of file names, so Markdown links to them work.
|
||
|
|
- A layout that renders UI, such as the header and footer, has its own doc: `layout.md` for the root layout, and `layout-<group>.md` for a route group's layout. A page doc links its layout doc and never repeats its rows. What crawlers get from a URL is its layout docs plus its page doc.
|
||
|
|
- A doc records decisions, never measurements: no sizes, timings, or dates, which change with every build. The PR plan records each key route's first-load JS and HTML size ([b2c-b2b.md](b2c-b2b.md#targets)).
|
||
|
|
- A rule for many pages is a decision ([decisions.md](decisions.md)), not a line in every doc.
|
||
|
|
|
||
|
|
## Vocabulary
|
||
|
|
|
||
|
|
- A section is a block of a page that a reader can name, such as the breadcrumb, the gallery, or the price. A part, in the rest of this skill, is a section with its own component; a part that loads on its own has its own `<Suspense>` and `PartBoundary`.
|
||
|
|
- An indexable page is a page whose robots value is `index`, or `index; noindex when <cause>`. Nothing above it streams: no `loading.tsx`, and no `<Suspense>` around `children` in a layout or in a component that a layout renders around them. A `noindex` or blocked page keeps its `loading.tsx` ([b2c-b2b.md](b2c-b2b.md#rendering)).
|
||
|
|
- An entity page is an indexable product, category, brand, seller, article, store branch, or campaign page: its URL names one record that can be missing or moved, so it answers a real 404 or 308 ([error-handling.md](error-handling.md#b2cb2b-server)).
|
||
|
|
- The first flush is the first bytes of the response: the shell, sent when everything outside a `<Suspense>` is ready; with `htmlLimitedBots`, it also waits for `generateMetadata`. On an indexable page, the shell holds everything outside the page's own `<Suspense>` parts, in place. A `<Suspense>` boundary that is complete by then is in place too, while the HTML of the shell, of the boundaries already in place, and its own stays within 12,800 bytes (React's `progressiveChunkSize`, which Next.js doesn't change), or when its own HTML is 500 bytes or less. Otherwise, its fallback is in place, and its content follows the footer, as for `streamed`.
|
||
|
|
- The First flush reads are the reads that the page's primary content needs. The page entry awaits them, `generateMetadata` reads the same caches when it needs data, and no Server Component outside a `<Suspense>` awaits any other read. A secondary section with a read of its own streams.
|
||
|
|
|
||
|
|
Each section gets three values. Use only these words, with these meanings. Each value has a code signal (Producing a doc from code).
|
||
|
|
|
||
|
|
**HTML**: what the response holds for the section, as a crawler reads it before any JavaScript runs. Crawlers get the same stream as browsers.
|
||
|
|
|
||
|
|
| Value | The server HTML holds |
|
||
|
|
|---|---|
|
||
|
|
| `first flush` | The real content, in the first flush, in document order. Every crawler reads it, whether or not it renders the page. |
|
||
|
|
| `streamed` | A skeleton in the first flush, and the real content later in the same response, in a hidden container at the end of `<body>`, such as `<div hidden id="S:n">` (`<table hidden>` for a part inside a table), until React's inline script moves it into place. It is the first content cut when the HTML passes Googlebot's 2 MB, and a slow part keeps the connection open, which uses up the site's crawl capacity. |
|
||
|
|
| `skeleton` | Only a same-size skeleton. The browser renders the real content, so Google sees it only after it renders the page, which is queued (seconds for most pages, hours for some) and not guaranteed, and then in the guest, first-visit state. |
|
||
|
|
| `none` | Nothing, such as a panel that mounts when it opens, a toast, or UI that a third-party script injects. |
|
||
|
|
|
||
|
|
**JS**: when the section's own client code loads. `next/link`, and the platform's `NavLink` that wraps it, count as `none`, because every page ships them.
|
||
|
|
|
||
|
|
| Value | The section's client code |
|
||
|
|
|---|---|
|
||
|
|
| `none` | None. The section is a Server Component, and nothing hydrates. |
|
||
|
|
| `hydrate` | In the route's first-load JavaScript, and it hydrates with the page. Both templates' client components ship to every device, so a `hydrate` row costs the other template too ([b2c-b2b.md](b2c-b2b.md#rendering)). |
|
||
|
|
| `lazy` | Its own chunk, fetched when the component first renders: `next/dynamic` in a client component. Only the device that renders it downloads it. |
|
||
|
|
| `visible` | Its own chunk, fetched when the section comes near the viewport: a `next/dynamic` component inside `LoadOnVisible`. |
|
||
|
|
| `interaction` | Its own chunk, fetched on intent and rendered on click: a facade, or `await import()` in an event handler. |
|
||
|
|
|
||
|
|
**Index**: what search engines need from the section. It limits the HTML value.
|
||
|
|
|
||
|
|
| Value | The section holds | Allowed HTML |
|
||
|
|
|---|---|---|
|
||
|
|
| `primary` | What the page is about, which crawlers must read without rendering: the H1; the main entity's name, price, availability, description, specs, and rating summary; the breadcrumb; the LCP image; and the links that are the only path to other pages, such as pagination and the category menu. The JSON-LD is primary too, and lives in its field | `first flush`, in both templates |
|
||
|
|
| `secondary` | Content that search engines should read but that the page doesn't depend on: related products, reviews, rails, brand strips, and blog teasers | `first flush` or `streamed`, in both templates |
|
||
|
|
| `no` | Content that search engines don't need: viewer-dependent UI, countdown digits, controls, and chat | Any value; it may differ between templates |
|
||
|
|
|
||
|
|
The Data column says where a section's data comes from: a server cache call (`getProduct(id)`), `query` for a browser query, `viewer` for the auth store and the viewer's queries, `clock` for the shared clock, `config` for constants and i18n text, or `—` for none.
|
||
|
|
|
||
|
|
When a developer calls a section "static", it means HTML `first flush` with JS `none`: server HTML that ships no JavaScript. It never means static rendering at build time, which these apps never use, because every page renders fresh per request ([rendering-patterns.md](rendering-patterns.md#rendering-patterns)). When a developer calls a section "dynamic", it means a section whose JS isn't `none`, never Next.js dynamic rendering, which every page uses. Docs write the values, never "static" or "dynamic". For example, "the home page is all static, and the discount timer has JS" becomes: most home sections are `first flush` and `none`; the timer's end time is `first flush`, inside its section; and its digits are a `↳ Countdown digits` row with `skeleton`, `hydrate`, and `no`.
|
||
|
|
|
||
|
|
Common combinations:
|
||
|
|
|
||
|
|
- `first flush` + `hydrate`: an interactive leaf rendered on the server, such as add to cart or a slider.
|
||
|
|
- `skeleton` + `hydrate`: viewer-dependent or time-dependent UI.
|
||
|
|
- `skeleton` + `lazy`: heavy UI that renders at once, in the browser only.
|
||
|
|
- `skeleton` + `visible`: a browser part below the fold.
|
||
|
|
- `first flush` + `interaction`: a facade whose widget loads on click.
|
||
|
|
|
||
|
|
## Doc format
|
||
|
|
|
||
|
|
Copy this into `docs/pages/<app>/<page>.md`. Keep a doc under about 40 lines.
|
||
|
|
|
||
|
|
```markdown
|
||
|
|
# <Page name>
|
||
|
|
|
||
|
|
`<route>` · `apps/<app>/src/app/<path>/page.tsx` · Type: <page type> · Layout: [<layout doc>](<layout doc>)
|
||
|
|
|
||
|
|
| Field | Value |
|
||
|
|
|---|---|
|
||
|
|
| Robots | `index`, `index; noindex when <cause>`, `noindex` (<where it's set>), or `blocked` (<the robots.ts rule>) |
|
||
|
|
| Status | `200`; `404` when <cause>; `308` when <cause> |
|
||
|
|
| loading.tsx | `none` (indexable page), or `<path>/loading.tsx` |
|
||
|
|
| First flush | <the First flush reads: the cache reads that the primary content needs>, or `none` |
|
||
|
|
| Canonical | <URL pattern>; drops <parameters> |
|
||
|
|
| H1 | <where its text comes from> |
|
||
|
|
| JSON-LD | <builder>: <node types>, or `none` |
|
||
|
|
| Scripts | <third-party scripts and their next/script strategy>, or `none` |
|
||
|
|
|
||
|
|
## Sections
|
||
|
|
|
||
|
|
In the mobile template's document order.
|
||
|
|
|
||
|
|
| Section | Data | HTML | JS | Index | Notes |
|
||
|
|
|---|---|---|---|---|---|
|
||
|
|
| <section> | <data> | <value> | <value> | <value> | <template differences, trigger, link targets> |
|
||
|
|
|
||
|
|
## Notes
|
||
|
|
|
||
|
|
- <what the tables can't say: accepted trade-offs, developer overrides, debt>
|
||
|
|
```
|
||
|
|
|
||
|
|
How to fill it:
|
||
|
|
|
||
|
|
- Header line: the route as users see it, the `page.tsx` path, the page type from [page-type-maps.md](page-type-maps.md), and the layout doc around the page.
|
||
|
|
- Robots: `index` for an indexable page; `index; noindex when <cause>` for one that turns `noindex` in one case, such as a category with no products, with where it's set; `noindex` and where it's set, such as the `(account)` layout's `metadata`; or `blocked` and its `robots.ts` rule. A blocked page sets no robots tag, because Google never fetches it to read one.
|
||
|
|
- Status: every status the URL answers, with its cause. An entity page lists its 404, and its 308 when its record can move or be replaced. A page with a `loading.tsx` answers 200, even when its data is missing.
|
||
|
|
- loading.tsx: `none` on every indexable page; the file that wraps a `noindex` or blocked page.
|
||
|
|
- First flush: the First flush reads (Vocabulary), which the page entry awaits. The layout's reads are in the layout doc. Write `none` when the page reads nothing before its first byte.
|
||
|
|
- Canonical: the pattern that `generateMetadata` gives `alternates.canonical`, and the parameters it drops, such as variant, sort, and tracking parameters.
|
||
|
|
- H1: where its text comes from. Each template has one H1, with the same text.
|
||
|
|
- JSON-LD: the builder from the platform's `app/json-ld/`, or the `buildGraph` call, and its node types; `none` on a `noindex` or blocked page. The page entry renders it before the template.
|
||
|
|
- Scripts: each `next/script` from `packages/integrations` that renders no section, such as analytics, with its strategy. A third-party widget with UI, such as chat, is a section.
|
||
|
|
- Sections: one row per section the page renders, in the mobile template's document order. The layout's sections are in the layout doc.
|
||
|
|
- Name each row after its component, so a search finds the code.
|
||
|
|
- A piece of a section that renders differently from the rest of it gets its own row, starting with `↳`.
|
||
|
|
- A row that only one template has names that template, `(desktop)` or `(mobile)`, and is Index `no`.
|
||
|
|
- Notes: how the templates differ (`mobile: tabs, every panel in the DOM`), what triggers a deferred section, where its links go, and `override:` with the reason when the developer asked for a placement that breaks a default.
|
||
|
|
- Notes: only what the tables can't say. Leave the heading out when there is nothing to say.
|
||
|
|
- A layout doc has the title `# Layout: <group>`, then one line: `` `<layout.tsx path>` · wraps every page in `<route group>` · its reads, <cache reads>, are in every page's first flush; a failed read answers 500 on every page under it ``. Then come the Sections table and a `Scripts:` line. It has no Field table, and the ready-for-review checks apply to its rows.
|
||
|
|
|
||
|
|
A doc is ready for review when:
|
||
|
|
|
||
|
|
- every `primary` row is `first flush` in both templates;
|
||
|
|
- every `secondary` row is `first flush` or `streamed` in both templates;
|
||
|
|
- the JSON-LD field's builder takes only config or the First flush reads, and nothing outside a `<Suspense>` awaits a read beyond the First flush reads;
|
||
|
|
- no `skeleton` or `none` row holds text or links that search engines need;
|
||
|
|
- every row that one template lacks is Index `no`;
|
||
|
|
- an indexable page has loading.tsx `none`, and an entity page lists its 404, and its 308 when its record can move or be replaced.
|
||
|
|
|
||
|
|
## Index file
|
||
|
|
|
||
|
|
`docs/pages/README.md` lists every doc, one line each, grouped by app: the route, the doc, the robots value, the JSON-LD nodes, and the browser-only sections, which are the rows whose HTML is `skeleton` or `none`. A layout's line names the layout and lists only its browser-only sections. Read the index file before a task that touches several pages, and open only the docs the task needs.
|
||
|
|
|
||
|
|
```markdown
|
||
|
|
# Page rendering docs
|
||
|
|
|
||
|
|
One line per route: route · doc · robots · JSON-LD · browser-only sections.
|
||
|
|
|
||
|
|
## shop
|
||
|
|
|
||
|
|
- `(shop)` layout · [layout-shop.md](shop/layout-shop.md) · — · — · browser: user menu, cart count, search suggestions
|
||
|
|
- `/` · [home.md](shop/home.md) · index · OnlineStore, WebSite · browser: countdown digits, recommended for you
|
||
|
|
- `/product/[id]` · [product-id.md](shop/product-id.md) · index · Product or ProductGroup, BreadcrumbList · browser: countdown digits, membership offer, price history, zoom viewer
|
||
|
|
- `/search` · [search.md](shop/search.md) · blocked · none · browser: none
|
||
|
|
```
|
||
|
|
|
||
|
|
## Writing a doc
|
||
|
|
|
||
|
|
Write a new page's doc in `docs/pages/<app>/` before the code, and link it from the plan. It is the plan's record of where each section renders and when its code loads.
|
||
|
|
|
||
|
|
1. Start from the page type's map in [page-type-maps.md](page-type-maps.md). Remove the sections the design doesn't have, and add the ones it has.
|
||
|
|
2. Give each section its Index value first, from what search engines need. Then pick its HTML and JS values with [rendering-patterns.md](rendering-patterns.md#choosing-where-a-part-loads). The Index value limits the HTML value.
|
||
|
|
3. Decide whether the page is indexable. An indexable page has no `loading.tsx` above it, and an entity page among them answers a real 404, and a 308 when its record can move ([error-handling.md](error-handling.md#b2cb2b-server)); a `noindex` or blocked page keeps its `loading.tsx`. Then fill the other fields.
|
||
|
|
4. Run the ready-for-review checks under Doc format, and change the design where one fails.
|
||
|
|
5. Add the page's line to `docs/pages/README.md`.
|
||
|
|
6. After the code is built, run Checking the crawler view. Where the doc and the HTML disagree, fix the code or the doc.
|
||
|
|
|
||
|
|
## Producing a doc from code
|
||
|
|
|
||
|
|
To write a missing doc, or to audit a page, read the code and predict each row, then prove the rows with Checking the crawler view. Where the code and the HTML disagree, the HTML wins, with one exception: a section that the page's own code wraps in `<Suspense>` is `streamed`, even when a response shows it in place, because a part whose read finishes before the first flush is written in place on that request only.
|
||
|
|
|
||
|
|
1. List the files that shape the route: its `page.tsx`, and every `layout.tsx`, `loading.tsx`, `error.tsx`, and `not-found.tsx` from its folder up to `app/`. A `loading.tsx` in this list wraps the page. So does a `<Suspense>` around `children` in a layout, or in a component that a layout renders around `children`, such as the `AppShell` or a provider.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
d='apps/<app>/src/app/(shop)/product/[id]'
|
||
|
|
while :; do ls "$d" | grep -E '^(page|layout|loading|error|not-found)\.tsx$' | sed "s|^|$d/|"; case "$d" in src/app|*/src/app|.|/) break;; esac; d=$(dirname "$d"); done
|
||
|
|
```
|
||
|
|
|
||
|
|
2. Read `page.tsx`:
|
||
|
|
- what the page entry awaits for its primary rows, and what `generateMetadata` awaits, give the First flush field; another read that the page entry awaits is a defect (below);
|
||
|
|
- where it calls `notFound()` and `permanentRedirect()`, and whether something from step 1 wraps it, give the Status and loading.tsx fields;
|
||
|
|
- the `metadata` or `generateMetadata` of the page and its layouts, and `robots.ts`, give the Robots and Canonical fields;
|
||
|
|
- the `<JsonLd>` element and its position give the JSON-LD field.
|
||
|
|
3. Walk each template's JSX in order. Each top-level organism is a row. Read each organism's files down to its leaves, and check each island or platform component it renders for `'use client'`: the leaves hold most of the signals in the table below. `grep -rlE "^['\"]use client" <organism folder>` lists the organism's own client files.
|
||
|
|
4. Record each finding below in the PR, or in the doc's Notes as debt.
|
||
|
|
|
||
|
|
| In the code | HTML | JS |
|
||
|
|
|---|---|---|
|
||
|
|
| A Server Component rendered from reads the page entry awaits, outside every `<Suspense>`, on an indexable page | `first flush` | `none` |
|
||
|
|
| On an indexable page, an async Server Component outside every `<Suspense>` that awaits a read beyond the First flush reads | `first flush`, which waits for that read | `none` |
|
||
|
|
| An async Server Component inside `<Suspense>`; or every row of a page under a `loading.tsx` once the shell's HTML and the page's pass 12,800 bytes | `streamed` | `none` |
|
||
|
|
| A `'use client'` component that a Server Component renders, with server props or hydrated query data | the value of the section around it | `hydrate` |
|
||
|
|
| `HydrationBoundary` with an awaited prefetch inside `<Suspense>`, `useSuspenseQuery` on an un-awaited prefetch, or a client component that reads an un-awaited server promise with `use()` inside `<Suspense>` | `streamed` | `hydrate` |
|
||
|
|
| Output decided by the auth store's `status`, by a `useQuery` with no server prefetch, or by `useClock` | `skeleton` | `hydrate` |
|
||
|
|
| Output rendered only after a mounted flag, a fetch in `useEffect`, or a `typeof window` check | `skeleton`, or `none` when it renders nothing until then | `hydrate` |
|
||
|
|
| `next/dynamic` with `ssr: false` | `skeleton` | `lazy` |
|
||
|
|
| `next/dynamic` without `ssr: false`, rendered at once | the value of the section around it | `lazy` |
|
||
|
|
| `LoadOnVisible` around a `next/dynamic` component | `skeleton` | `visible` |
|
||
|
|
| A facade that imports its widget on intent or click | `first flush`, the facade | `interaction` |
|
||
|
|
| A panel rendered only when open (`{isOpen && <Panel />}`), or content fetched on click | `none` | the trigger's value |
|
||
|
|
| `<details>`, or tabs and accordions that render every panel and hide the closed ones (island's `TabsPanel` with `forceMount`) | the value of the section around it | the toggle's value |
|
||
|
|
| `next/script` from `packages/integrations` | the Scripts field | the Scripts field |
|
||
|
|
|
||
|
|
Each finding below is a defect. Fix it, or record it as debt.
|
||
|
|
|
||
|
|
- An indexable page with a `loading.tsx` above it, or a `<Suspense>` around `children` in a layout or in a component a layout renders around them: an entity page answers 200 for a missing entity, at any size, and once the HTML passes React's 12,800-byte limit, the page's content follows the footer in a hidden `<div>`.
|
||
|
|
- On an indexable page, the page entry or a Server Component outside every `<Suspense>` that awaits a read beyond the First flush reads: the status line and the first byte wait for that read. Move it into a part inside `<Suspense>` and `PartBoundary`.
|
||
|
|
- JSON-LD in a layout, a template, a part inside `<Suspense>`, a client component, or `next/script`.
|
||
|
|
- A `primary` row that isn't `first flush`: inside a `<Suspense>` in the page's own code, in a `useQuery` part, behind `ssr: false`, inside `LoadOnVisible`, or behind a click. A `secondary` row that is `skeleton` or `none`.
|
||
|
|
- Content that search engines need kept out of the server HTML by a mounted flag, a fetch in `useEffect`, or a `typeof window` check: render it on the server, or use a technique from [rendering-patterns.md](rendering-patterns.md#choosing-where-a-part-loads).
|
||
|
|
- A tab, accordion, drawer, or slider that mounts a `primary` or `secondary` panel, or its links, only when open, or fetches them on click.
|
||
|
|
- Navigation through `onClick` with `router.push`, a `<button>`, or a clickable `<div>`; menu links fetched on hover or tap; page state, or an indexable filter's state, in a `#fragment`.
|
||
|
|
- A `primary` or `secondary` section, heading, link, or image that only one template has, or a different H1 in each template.
|
||
|
|
- A heavy client component that only one template renders, imported statically: every device downloads it. Load it through `next/dynamic` in a client component.
|
||
|
|
- `LoadOnVisible` around a statically imported client component: only its render waits, and its code is in the first-load JavaScript ([rendering-patterns.md](rendering-patterns.md#loading-patterns)).
|
||
|
|
- `isBot`, or a list of bots, in a rendering decision.
|
||
|
|
- `Date.now()` or `new Date()` in a client component's render; date formatting in a client component whose output the server renders; `suppressHydrationWarning` on time text; a `setInterval` per component.
|
||
|
|
- Robots tags, the canonical, or the title set or changed by client code.
|
||
|
|
- A locale or a currency on an indexable page taken from a cookie, `Accept-Language`, or the IP.
|
||
|
|
- Products reachable only through search or "Load more".
|
||
|
|
- HTML or first-load JS over its target ([b2c-b2b.md](b2c-b2b.md#targets)).
|
||
|
|
|
||
|
|
## Checking the crawler view
|
||
|
|
|
||
|
|
Check a production build, locally or on staging: build with the app's `build` target and run its `start` target (`nx show project <app>` lists them). The dev server's HTML differs from production. Run the checks for both templates, on the page of this type with the most content, in a new, empty folder.
|
||
|
|
|
||
|
|
1. Fetch the page as both Googlebots. Without `--compressed`, curl gets the uncompressed HTML, which is what Googlebot's 2 MB limit counts. The two byte counts together are the template's HTML size: check it against the HTML target ([b2c-b2b.md](b2c-b2b.md#targets)).
|
||
|
|
|
||
|
|
```bash
|
||
|
|
BASE='http://localhost:3000'
|
||
|
|
URL="$BASE/product/123"
|
||
|
|
STAGING_URL='https://<staging host>/product/123'
|
||
|
|
MOBILE='Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.7390.122 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
|
||
|
|
DESKTOP='Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/141.0.7390.122 Safari/537.36'
|
||
|
|
curl -s -A "$MOBILE" -o mobile.html -w 'mobile: %{http_code}, %{size_header} + %{size_download} bytes\n' "$URL"
|
||
|
|
curl -s -A "$DESKTOP" -o desktop.html -w 'desktop: %{http_code}, %{size_header} + %{size_download} bytes\n' "$URL"
|
||
|
|
```
|
||
|
|
|
||
|
|
2. Split each response into what the first flush holds and what streams, with the scripts removed. `S` matches every hidden container that React streams into. The `.txt` files hold each part's text with React's `<!-- -->` separators removed and its HTML escapes decoded, so a search for a value the page shows matches.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
S='<(?:div hidden |table hidden(?: |><(?:tbody|tr|colgroup) )|svg aria-hidden="true" style="display:none" |math aria-hidden="true" style="display:none" )id="S:'
|
||
|
|
D='s/<!-- -->//g; s/'/\x27/g; s/"/"/g; s/</</g; s/>/>/g; s/&/&/g'
|
||
|
|
for t in mobile desktop; do
|
||
|
|
perl -0pe 's/<script\b.*?<\/script>//gs; s/'"$S"'.*//s' $t.html > $t.first-flush.html
|
||
|
|
perl -0ne 'print $1 if /('"$S"'.*)/s' $t.html | perl -0pe 's/<script\b.*?<\/script>//gs' > $t.streamed.html
|
||
|
|
perl -0ne 'print $1 if /<body\b[^>]*>(.*)/s' $t.first-flush.html | perl -pe "$D" > $t.first-flush.txt
|
||
|
|
perl -pe "$D" $t.streamed.html > $t.streamed.txt
|
||
|
|
done
|
||
|
|
```
|
||
|
|
|
||
|
|
3. Check the rows. Every `primary` row's text is in `*.first-flush.txt` of both templates, and every `secondary` row's text in `*.first-flush.txt` or `*.streamed.txt`. Search for one value that the page shows, such as `grep -c '<product name>' mobile.first-flush.txt desktop.first-flush.txt`. Each file is one line, so each count is 0 or 1.
|
||
|
|
|
||
|
|
4. Head, H1, and JSON-LD. Both templates have the same title, description, canonical, and robots tags, one H1 with the same text, and the same JSON-LD element. The diff proves only that the templates match, and the H1 count proves that each has one. The first offset that the last command prints must be the JSON-LD element, so it is in the first flush.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
for t in mobile desktop; do
|
||
|
|
perl -0ne 'print "$1\n" if /<head>(.*?)<\/head>/s' $t.html | grep -o '<title>[^<]*</title>\|<meta name="description"[^>]*>\|<link rel="canonical"[^>]*>\|<meta name="robots"[^>]*>' > $t.head
|
||
|
|
perl -0ne 'print "$1\n" while /<h1\b[^>]*>(.*?)<\/h1>/gs' $t.first-flush.html | sed 's/<[^>]*>//g' >> $t.head
|
||
|
|
perl -0ne 'print "$1\n" while /<script type="application\/ld\+json">(.*?)<\/script>/gs' $t.html >> $t.head
|
||
|
|
echo "$t: $(grep -o '<h1\b' $t.first-flush.html | wc -l) H1" # 1 each
|
||
|
|
done
|
||
|
|
diff mobile.head desktop.head && cat mobile.head
|
||
|
|
grep -boP '<script type="application/ld\+json"|'"$S" mobile.html | head -2
|
||
|
|
```
|
||
|
|
|
||
|
|
Then run the page through the Rich Results Test as Googlebot Smartphone: in URL mode on a staging host that the internet can reach, or in code mode with `mobile.html` for a local build. For the home page, use the Schema Markup Validator ([lib-json-ld.md](lib-json-ld.md#testing-and-monitoring)).
|
||
|
|
|
||
|
|
5. Links and images. Both templates have the same `<a href>` links, image URLs, and alt texts, in the first flush or streamed; step 3 proves which ones are in the first flush. Each line that `diff` prints is a link or an image that only one template has, which only an Index `no` row may have. The `sed` drops `next/image`'s width and quality, which change with each template's image size.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
for t in mobile desktop; do
|
||
|
|
cat $t.first-flush.html $t.streamed.html | grep -o '<a [^>]*href="[^"#][^"]*"' | grep -o 'href="[^"]*"' | sort -u > $t.links
|
||
|
|
cat $t.first-flush.html $t.streamed.html | grep -o '<img [^>]*>' | grep -o ' \(src\|alt\)="[^"]*"' | sed 's/&w=[0-9]*&q=[0-9]*//' | sort -u >> $t.links
|
||
|
|
done
|
||
|
|
diff mobile.links desktop.links
|
||
|
|
```
|
||
|
|
|
||
|
|
6. Status, `robots.txt`, and `Vary`. On an entity page, a missing entity answers 404, and a replaced one 308 with its target. No `Disallow` line matches an indexable page, its variant URLs, or `/_next/static/`; a blocked page matches its own rule, such as `Disallow: /search`. On staging, behind the reverse proxy or the CDN and never on a local start, every HTML response's `Vary` includes `User-Agent`.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
curl -s -o /dev/null -A "$MOBILE" -w '%{http_code} %{redirect_url}\n' "$BASE/product/does-not-exist" # 404
|
||
|
|
curl -s -o /dev/null -A "$MOBILE" -w '%{http_code} %{redirect_url}\n' "$BASE/product/<replaced-id>" # 308 <replacement URL>
|
||
|
|
curl -s "$BASE/robots.txt" | grep -i '^disallow'
|
||
|
|
curl -s -o /dev/null -D - -A "$MOBILE" "$STAGING_URL" | grep -i '^vary:' # includes User-Agent
|
||
|
|
```
|
||
|
|
|
||
|
|
7. States. For each case that the doc's Status, Robots, and Canonical fields name, set `URL` to one URL of that case, and run steps 1 to 4 again in a new, empty folder. Then check: a variant URL (200; its price in the first flush; a canonical without the variant parameters), an out-of-stock product (200; its stock text in the first flush; `OutOfStock` in the JSON-LD), `?page=2` of a listing (200; its own canonical; `<a href>` links to the previous and next pages), and a page past the last or an empty combination of indexable filters (404 for both user agents in step 1; skip steps 2 to 4).
|
||
|
|
|
||
|
|
8. In the browser. Open the page in Chrome, with the Googlebot Smartphone user agent from DevTools' Network conditions. The console shows no React error. Production reports a text mismatch, such as a clock read during render, as `Minified React error #418`. It reports a missing `getServerSnapshot` inside a `<Suspense>` as `#419`, and the server HTML then holds only that boundary's fallback. Outside every `<Suspense>`, as for a section in an indexable page's first flush, a missing `getServerSnapshot` fails the whole response instead: a 500 with `noindex` and an empty `<body>`, which step 1's status shows, and no React error in the console. Then turn JavaScript off and reload. What still shows is the `first flush` content; streamed sections keep their skeletons, although their HTML is in the response.
|
||
|
|
|
||
|
|
9. JavaScript per route. `next build` stopped printing first-load JS in Next.js 16. Next.js 16.4 writes each route's first-load chunks to `.next/diagnostics/route-bundle-stats.json`, an undocumented file, so check that it still exists after an upgrade. Run this where the build's `.next/` folder is. Route-specific JS is the first-load JS minus the chunks that every route shares.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
node -e '
|
||
|
|
const fs = require("fs"), zlib = require("zlib");
|
||
|
|
const stats = require("./.next/diagnostics/route-bundle-stats.json");
|
||
|
|
const gz = (f) => zlib.gzipSync(fs.readFileSync(f)).length;
|
||
|
|
const shared = stats.map((r) => r.firstLoadChunkPaths).reduce((a, b) => a.filter((f) => b.includes(f)));
|
||
|
|
for (const { route, firstLoadChunkPaths: c } of stats) console.log(route.padEnd(40), (c.reduce((s, f) => s + gz(f), 0) / 1024).toFixed(1), "KB first-load,", (c.filter((f) => !shared.includes(f)).reduce((s, f) => s + gz(f), 0) / 1024).toFixed(1), "KB route-specific (gzip)");'
|
||
|
|
```
|
||
|
|
|
||
|
|
The file leaves out the chunk of a `next/dynamic` component that a template renders at once, which that template's HTML preloads. So also measure each template from step 1's HTML, and check the larger number against the target. In the check folder, list each template's chunks; the pattern skips the `noModule` polyfills and accepts the `?dpl=` that `NEXT_DEPLOYMENT_ID` adds:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
for t in mobile desktop; do perl -0ne 'print ".next/$1\n" while /<(?:script src|link rel="preload"[^>]*href)="\/_next\/(static\/chunks\/[^"?]+\.js)(?:\?[^"]*)?"(?![^>]*noModule)/g' $t.html | sort -u > $t.chunks; done
|
||
|
|
```
|
||
|
|
|
||
|
|
Then, where the build's `.next/` folder is:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
node -e 'const fs = require("fs"), zlib = require("zlib"); for (const list of process.argv.slice(1)) console.log(list, (fs.readFileSync(list, "utf8").trim().split("\n").reduce((s, f) => s + zlib.gzipSync(fs.readFileSync(f)).length, 0) / 1024).toFixed(1), "KB gzip")' <check folder>/mobile.chunks <check folder>/desktop.chunks
|
||
|
|
```
|
||
|
|
|
||
|
|
`next analyze` shows what a route's client JavaScript holds and why each module is there, and `next analyze --output` writes the same data to `.next/diagnostics/analyze` without a server. Its route totals also count the `noModule` polyfills, which modern browsers skip, and use their own compressed size, so compare only the numbers above with the target.
|
||
|
|
|
||
|
|
After a release, Search Console shows what Google got. URL Inspection's live test gives the rendered HTML, a screenshot, and the console messages, as Googlebot Smartphone. In the Page indexing report, a rise in "Soft 404" or "Excluded by 'noindex' tag" for an indexable page type means a template lost its content or answers the wrong status. Crawl stats give the response time and the status per response code.
|
||
|
|
|
||
|
|
## Keeping docs current
|
||
|
|
|
||
|
|
- Update the doc in the same PR as any change to the page's sections, their data reads, their HTML, JS, or Index values, or the page's robots, status, canonical, JSON-LD, or scripts. A change inside a section that keeps its data read and its HTML, JS, and Index values needs no doc change.
|
||
|
|
- Update the layout doc when the header or footer changes, and the index line when the robots value, the JSON-LD, or the browser-only sections change.
|
||
|
|
- A route without a doc gets one in the first PR that changes it. Produce it from code, in its own commit.
|
||
|
|
- Read a page's doc before you change the page. It says where each section renders, without opening every file.
|
||
|
|
- The reviewer reads the doc's diff first, runs the ready-for-review checks, and checks that the code diff matches the doc. A rendering change without its doc change blocks the merge ([definition-of-done.md](definition-of-done.md#review)).
|
||
|
|
- SEO and QA read the docs and the index file for the crawler view of every page.
|
||
|
|
|
||
|
|
## Example: home page
|
||
|
|
|
||
|
|
A Digikala-like shop. The header and footer live in `(shop)/layout.tsx`, so `layout-shop.md` documents them once, and the home doc links it.
|
||
|
|
|
||
|
|
`docs/pages/shop/layout-shop.md`:
|
||
|
|
|
||
|
|
```markdown
|
||
|
|
# Layout: shop
|
||
|
|
|
||
|
|
`apps/shop/src/app/(shop)/layout.tsx` · wraps every page in `(shop)` · its reads, `getMenu()`, are in every page's first flush; a failed read answers 500 on every page under it
|
||
|
|
|
||
|
|
| Section | Data | HTML | JS | Index | Notes |
|
||
|
|
|---|---|---|---|---|---|
|
||
|
|
| Logo | config | first flush | none | primary | `<a href="/">`; its alt text is the store's name |
|
||
|
|
| SearchBox | — | first flush | hydrate | no | a GET form to `/search`, so it works before hydration |
|
||
|
|
| ↳ SearchSuggestions | query | none | interaction | no | the code loads on focus; suggestions load from the second typed character, debounced 300 ms |
|
||
|
|
| CategoryMenu | `getMenu()` | first flush | hydrate | primary | desktop: a mega menu that opens on hover; mobile: a drawer; both keep every category's `<a href>` in the DOM while closed |
|
||
|
|
| UserMenu | viewer | skeleton | hydrate | no | guest: a login link; member: the account menu |
|
||
|
|
| CartCount | viewer | skeleton | hydrate | no | the badge keeps its box while the viewer is unknown |
|
||
|
|
| BottomNavigation (mobile) | config | first flush | none | no | home, categories, cart, and account |
|
||
|
|
| FooterLinks | `getMenu()`, config | first flush | none | primary | top categories, store branches, policies, about, and contact |
|
||
|
|
| NewsletterForm | — | first flush | hydrate | no | |
|
||
|
|
| ChatButton | — | first flush | interaction | no | the chat integration's facade; the widget loads on click |
|
||
|
|
|
||
|
|
Scripts: analytics, `next/script` with `lazyOnload`.
|
||
|
|
```
|
||
|
|
|
||
|
|
`docs/pages/shop/home.md`:
|
||
|
|
|
||
|
|
```markdown
|
||
|
|
# Home
|
||
|
|
|
||
|
|
`/` · `apps/shop/src/app/(shop)/page.tsx` · Type: home · Layout: [layout-shop.md](layout-shop.md)
|
||
|
|
|
||
|
|
| Field | Value |
|
||
|
|
|---|---|
|
||
|
|
| Robots | `index` |
|
||
|
|
| Status | `200` |
|
||
|
|
| loading.tsx | `none`: links to `/` are `NavLink`s |
|
||
|
|
| First flush | `getHomePage()`, which the page entry awaits; the metadata is fixed, in the `metadata` export |
|
||
|
|
| Canonical | `/`; drops tracking parameters |
|
||
|
|
| H1 | `home:title` from i18n, in the hero section |
|
||
|
|
| JSON-LD | `buildOrganization(SEO_ORGANIZATION)` and `buildWebSite(SEO_WEBSITE)` from `configs/seo.ts`: `OnlineStore`, `WebSite` |
|
||
|
|
| Scripts | none beyond the layout's |
|
||
|
|
|
||
|
|
## Sections
|
||
|
|
|
||
|
|
In the mobile template's document order.
|
||
|
|
|
||
|
|
| Section | Data | HTML | JS | Index | Notes |
|
||
|
|
|---|---|---|---|---|---|
|
||
|
|
| HeroBanners, with the H1 | `getHomePage()` | first flush | hydrate | primary | every slide's `<a href>` and `<img>` in the DOM; the first image eager with high fetch priority, the others lazy; the slider leaf only moves the slides |
|
||
|
|
| CategoryTiles | `getHomePage()` | first flush | none | primary | an `<a href>` per top category; mobile: one scrolling row; desktop: a grid |
|
||
|
|
| OffersRail | `getHomePage()` | first flush | none | secondary | 20 cards, each an `<a href>` with a name, image, price, struck-through price, and discount; the header says when the offers end, in a `<time>` formatted on the server |
|
||
|
|
| ↳ Countdown digits | clock | skeleton | hydrate | no | one `Countdown` in the rail's header, not one per card; at zero its ended label replaces the digits, and nothing refreshes |
|
||
|
|
| CampaignBanners | `getHomePage()` | first flush | none | primary | an `<a href>` to each campaign's permanent URL, the only links to campaign pages |
|
||
|
|
| BestsellerRails | `getBestsellers(categoryId)` | streamed | none | secondary | three rails, each in its own `<Suspense>` and `PartBoundary`; category pagination reaches the same products |
|
||
|
|
| RecommendedForYou | viewer | skeleton | visible | no | members: personal picks; guests, and Google: popular products |
|
||
|
|
| BrandStrip | `getHomePage()` | first flush | none | secondary | an `<a href>` per brand page |
|
||
|
|
| BlogTeasers | `getHomePage()` | first flush | none | primary | an `<a href>` per latest article, the only links into the blog from outside it |
|
||
|
|
|
||
|
|
## Notes
|
||
|
|
|
||
|
|
- Desktop shows the offers rail beside the hero, and mobile shows it below the category tiles. Both templates hold the same sections.
|
||
|
|
- The rail's offers end together, so the rail has one countdown. On each product page, the same `discountEndsAt` is the offer's sale end in the JSON-LD and the product countdown's `endsAt`.
|
||
|
|
```
|
||
|
|
|
||
|
|
How the example meets the rules:
|
||
|
|
|
||
|
|
- The header and footer are documented once, and their primary rows are in every page's first flush.
|
||
|
|
- The home page is indexable, so nothing above it streams: its primary content and JSON-LD are in the shell, in place, whatever the page's size ([b2c-b2b.md](b2c-b2b.md#rendering)). A click on a link to `/`, a `NavLink`, shows its pending hint until the page arrives.
|
||
|
|
- The discount timer's end time is server HTML. Only its digits are a browser leaf, and they are never indexable content ([rendering-patterns.md](rendering-patterns.md#time-dependent-ui)).
|