@ogygia/testing

This page is generated from the package’s own .d.ts files on every build, so the signatures, descriptions and examples below always match the source.

import { … } from '@ogygia/testing';

Exports checkBudget · compare · createOgygia · forAttachment · playwrightDriver · toMarkdown · turnOnDevtools · withOgygia · ogygiaMatchers · Budget · CompareOptions · HeldNavigation · MatcherResult · Ogygia · OgygiaDriver · OgygiaEngine · OgygiaHydrationReport · OgygiaOptions · OgygiaPageReport · OgygiaProfileReport · RunDiff · CompareInput · OgygiaReport · OgygiaSessionReport · OgygiaTestOptions · SessionMoveKey

checkBudget

function

Each line of a budget against a report: { key, measured, limit, ok }.

function checkBudget(r: OgygiaReport, b: Budget): {
  key: string;
  measured: number | null;
  limit: number;
  ok: boolean;
}[];

compare

function

What changed between two runs of the same tests (each: the ogygia.json bodies, parsed).

A test can carry several runs a side ({ test, runs }): real pages are noisy, and one run against one run turns that noise into false regressions. Then

• load signals — page vitals and findings, hydration, the profile's score, render time and JS — are compared on the runs' median (a finding or a broken island counts when most runs have it);
• session signals — what a scripted session ran into: its findings, errors, shifts, slow interactions, blocked navigations — count only when EVERY head run has it and NO base run does (a count: when every head run is above every base run, or below), since they depend on the path the session happened to take. One run a side is the same rules on a single sample.

function compare(baseRun: CompareInput[], headRun: CompareInput[], opts?: CompareOptions): RunDiff[];

createOgygia

function

The primitives over any driver.

function createOgygia(driver: OgygiaDriver, options?: OgygiaOptions): Ogygia;

forAttachment

function

A report as kept for a comparison (a profile's full JSON is large).

function forAttachment(r: OgygiaReport): unknown;

playwrightDriver

function

The driver withOgygia builds, for a Playwright Page you hold yourself (several contexts in one test, a custom fixture): createOgygia(playwrightDriver(page, { baseURL })). onNavigationHeld is called as each document load is held back, so you can note what your script was doing then.

function playwrightDriver(page: Page, opts?: {
  baseURL?: string;
  onNavigationHeld?: (nav: HeldNavigation) => void;
}): OgygiaDriver;

toMarkdown

function

A markdown summary of compare's answer — the body of a PR comment.

function toMarkdown(diffs: RunDiff[]): string;

turnOnDevtools

function

Turn devtools on for this page's browser context, so a build measures it from its first load (the og_devtools=on cookie): for baseURL, or with none for the origin of the first page it opens, on that very request (Playwright's route handlers run newest first, so the test's own still run after).

function turnOnDevtools(page: Page, baseURL?: string, opts?: {
  lean?: boolean;
}): Promise<void>;

withOgygia

function

Your Playwright test with an ogygia fixture, and your expect with ogygia's matchers. Options per project: use: { ogygiaOptions: { profilerKey, attach: false } }.

function withOgygia<TA extends PlaywrightTestArgs & PlaywrightTestOptions, WA extends PlaywrightWorkerArgs & PlaywrightWorkerOptions>(test: TestType<TA, WA>, expect: Expect): {
  test: TestType<TA & {
    ogygia: Ogygia;
    ogygiaOptions: OgygiaTestOptions;
  }, WA>;
  expect: Expect;
};

ogygiaMatchers

const

expect.extend(ogygiaMatchers) — Playwright, Vitest and Jest all take this shape.

const ogygiaMatchers: {
  toHaveNoFindings(received: OgygiaReport, opts?: {
    severity?: Severity;
    ignore?: string[];
  }): MatcherResult;
  toMeetBudget(received: OgygiaReport, budget: Budget): MatcherResult;
  toHydrateCleanly(received: OgygiaReport, opts?: {
    allow?: HydrationStatus[];
    allowAttributeChanges?: boolean;
    allowInput?: boolean;
  }): MatcherResult;
}

Budget

interface

interface Budget {/*…*/}

Budget.lcp

lcp?: number;

Budget.fcp

fcp?: number;

Budget.cls

cls?: number;

Budget.inp

inp?: number;

Budget.ttfb

ttfb?: number;

Budget.longTaskMs

longTaskMs?: number;

Budget.islandWakeMs

the slowest island's wake to hydrated, ms

islandWakeMs?: number;

Budget.maxInteractionMs

maxInteractionMs?: number;

Budget.slowInteractions

slowInteractions?: number;

Budget.errors

errors?: number;

Budget.failedRequests

failedRequests?: number;

Budget.serverBusyMs

with record(…, { server: true }): the server's busy CPU over the session, ms

serverBusyMs?: number;

Budget.score

score?: number;

Budget.renderMs

renderMs?: number;

Budget.jsBytes

jsBytes?: number;

CompareOptions

interface

How far a vital must move to count, both ways (a run's own noise): an absolute floor, and for timings a share of the base too — { lcp: 150 } = 150 ms and 10 %. Defaults: lcp/fcp 150, ttfb 100, inp 50, cls 0.02.

interface CompareOptions {/*…*/}

CompareOptions.noise

noise?: Partial<Record<"lcp" | "fcp" | "ttfb" | "inp" | "cls", number>>;

HeldNavigation

interface

A document load ogygia held back (record(…, { blockNavigation: true })), t on the page's clock.

interface HeldNavigation {/*…*/}

HeldNavigation.url

url: string;

HeldNavigation.t

t: number | null;

MatcherResult

interface

What each matcher answers — the shape Playwright, Vitest and Jest all read.

interface MatcherResult {/*…*/}

MatcherResult.pass

pass: boolean;

MatcherResult.name

name: string;

MatcherResult.message

message: () => string;

Ogygia

interface

interface Ogygia {/*…*/}

Ogygia.page

page(): Promise<OgygiaPageReport>;

Ogygia.hydration

hydration(): Promise<OgygiaHydrationReport>;

Ogygia.record

server: true also records the server over the same stretch (needs devtools({ profiler }), the devtools plugin's profiler). blockNavigation: true holds back any document load the actions cause (a reload, a link), so the page and the recording stay, and lists each in navigations (needs a driver that can: Playwright's)

record(actions: () => Promise<unknown>, opts?: {
    server?: boolean;
    blockNavigation?: boolean;
  }): Promise<OgygiaSessionReport>;

Ogygia.profile

fields: only these top-level sections of the report (with what names it — schema, engine, id, target, links), e.g. ['requests', 'network', 'findings', 'assets', 'summary']: a full report is hundreds of KB, mostly stacks, hot_functions and drill

profile(path: string, opts?: {
    runs?: number;
    fields?: string[];
  }): Promise<OgygiaProfileReport>;

Ogygia.settled

wait until no island is waking or hydrating

settled(opts?: {
    quiet?: number;
    timeout?: number;
  }): Promise<boolean>;

Ogygia.reports

every report taken so far, in order

readonly reports: OgygiaReport[];

OgygiaDriver

interface

What ogygia needs from a page driver. Playwright's Page fits as is (withOgygia wires it); Puppeteer's too (evaluate, and fetch for the profiler).

interface OgygiaDriver {/*…*/}

OgygiaDriver.evaluate

run fn(arg) in the page and return its (JSON) result. Loosely typed (unknown in, unknown out, and a method, so its parameters compare both ways), so page.evaluate of any driver fits without a cast

evaluate(fn: (arg: unknown) => unknown, arg?: unknown): Promise<unknown>;

OgygiaDriver.get

GET a URL relative to the app (for profile()); default: fetch inside the page

get?(url: string, headers: Record<string, string>): Promise<{
    status: number;
    body: string;
  }>;

OgygiaDriver.post

POST to a URL relative to the app (the server window of record(…, { server: true })); default: fetch inside the page

post?(url: string, headers: Record<string, string>): Promise<{
    status: number;
    body: string;
  }>;

OgygiaDriver.wait

wait ms (default: a timer)

wait?(ms: number): Promise<void>;

OgygiaDriver.blockNavigation

start holding back the page's document loads (answer them 204, so the page stays) for record(…, { blockNavigation: true }); stop() ends it and lists them, t on the page's clock

blockNavigation?(): Promise<{
    stop(): Promise<{
      url: string;
      t: number | null;
    }[]>;
  }>;

OgygiaDriver.tagRequests

add header: value to every request the page makes to its own origin until the returned function is called — the server window's id, so each instance of a serverless deployment attributes the session's requests (record(…, { server: true }))

tagRequests?(header: string, value: string): Promise<() => Promise<void>>;

OgygiaEngine

interface

The page's testing engine that answered: it ships with the SITE's build, so it can be older than this package (version 1: 0.8.2, before keys, navigations, finding names and counts.cls).

interface OgygiaEngine {/*…*/}

OgygiaEngine.version

version: number;

OgygiaEngine.lean

the page ran lean testing (og_testing=lean): the testing API and nothing else of devtools. Absent from an engine before version 3 (it ran full devtools whatever the cookie said)

lean?: boolean;

OgygiaHydrationReport

interface

interface OgygiaHydrationReport {/*…*/}

OgygiaHydrationReport.kind

kind: "hydration";

OgygiaHydrationReport.measured

measured: boolean;

OgygiaHydrationReport.engine

engine: OgygiaEngine;

OgygiaHydrationReport.islands

key: the instance — its islandKey attribute (the nearest, across shadow roots), else Name#n in document order. Absent from an engine before version 2 (compare then tallies by name).

islands: {
    fp: string;
    key?: string;
    name: string;
    wake: string;
    status: HydrationStatus;
    reason: string;
    diff: HtmlDiff | null;
    t0?: number | null;
    done?: number | null;
    attributesOnly?: boolean;
    attributes?: {
      tag: string;
      name: string;
      from: string | null;
      to: string | null;
    }[] | null;
    inputDuringHydration?: boolean;
    ignored?: {
      tag: string;
      name: string;
      from: string | null;
      to: string | null;
    }[];
  }[];

OgygiaOptions

interface

interface OgygiaOptions {/*…*/}

OgygiaOptions.profilerKey

the profiler's key, for profile() on a build (a dev server needs none)

profilerKey?: string;

OgygiaOptions.profilerBase

where the profiler is mounted (default /__profiler)

profilerBase?: string;

OgygiaOptions.timeout

how long to wait for the page's testing API (default 15 s)

timeout?: number;

OgygiaOptions.islandKey

the attribute that names an island INSTANCE in hydration() (on the island or an ancestor: an id your app renders, e.g. a CMS block's). Without it, or where it is missing: Name#n

islandKey?: string;

OgygiaOptions.ignoreAttributes

attribute changes hydration() leaves out: an island whose every difference matches reads clean (with what it ignored in ignored). Each rule: the element's tag, the attribute's name, its new value (or old, when removed) — a string matches exactly, a RegExp by test. E.g. web components setting their own as they upgrade: [{ tag: /^qds-/, name: /^(tabindex|aria-)/ }, { name: 'style', value: '' }]

ignoreAttributes?: {
    tag?: string | RegExp;
    name: string | RegExp;
    value?: string | RegExp;
  }[];

OgygiaOptions.cpuFunctions

how many of the page load's hottest functions page().cpu.fns keeps (default 15, up to 500)

cpuFunctions?: number;

OgygiaPageReport

interface

interface OgygiaPageReport {/*…*/}

OgygiaPageReport.kind

kind: "page";

OgygiaPageReport.engine

engine: OgygiaEngine;

OgygiaPageReport.self

what MEASURING cost this page, which no visitor with devtools off pays — subtract it: script_ms, the main-thread time of ogygia's own measuring so far (by phases); bytes, the decoded size of the testing engine's files. Engine 3

self?: {
    script_ms: number;
    bytes: number;
    phases: Record<string, number>;
  };

OgygiaPageReport.url

url: string;

OgygiaPageReport.route

the SvelteKit route id the server rendered (/[lang]/about-us), from a build with devtools on; null on the dev server or an engine before 3

route?: string | null;

OgygiaPageReport.nav

nav: {
    to: string;
    t: number;
  } | null;

OgygiaPageReport.vitals

vitals: RatedVital[];

OgygiaPageReport.islands

islands: IslandRow[];

OgygiaPageReport.findings

findings: (PageFinding & {
    names: string[];
  })[];

OgygiaPageReport.blocking

blocking: {
    url: string;
    ms: number;
    bytes: number;
  }[];

OgygiaPageReport.bytes

bytes: {
    type: string;
    count: number;
    transfer: number;
    size: number;
  }[];

OgygiaPageReport.longtask_ms

longtask_ms: number;

OgygiaPageReport.cpu

cpu: {
    busy_ms: number;
    by_kind: {
      kind: string;
      ms: number;
    }[];
    fns: {
      name: string;
      file: string;
      line: number | null;
      self_ms: number;
      total_ms: number;
    }[];
  } | null;

OgygiaPageReport.unmeasured

what the test's browser cannot measure (WebKit: layout shifts, long tasks, interaction timing, the CPU sampler): findings that need them cannot appear, whatever the page does

unmeasured: string[];

OgygiaProfileReport

interface

The profiler's JSON answer (/__profiler/page?format=json), the fields a test reads first typed.

interface OgygiaProfileReport {/*…*/}

OgygiaProfileReport.kind

kind: "profile";

OgygiaProfileReport.engine

the ogygia that produced it (the page reports' engine.version): a profiler from before 3 sends none — its findings are 0.8.2's, so compare profiles only across one version

engine?: {
    version: number;
  };

OgygiaProfileReport.id

id: string;

OgygiaProfileReport.target

target: {
    page: string;
    runs: number[];
  };

OgygiaProfileReport.score

score: {
    score: number;
    grade: string;
    categories: {
      key: string;
      label: string;
      score: number;
      value: string;
      lost?: number;
    }[];
    missing: {
      key: string;
      label: string;
      why: string;
    }[];
  } | null;

OgygiaProfileReport.forecast

forecast: {
    now_ms: number;
    after_ms: number;
  } | null;

OgygiaProfileReport.findings

findings: {
    severity: string;
    code: string;
    message: string;
    fix?: string;
  }[];

OgygiaProfileReport.assets

the page's weight (a build), or { missing } why it was not weighed (a dev server)

assets: {
    totals?: {
      js: number;
      wire: number;
      lazy_js: number;
    };
    missing?: string;
    files?: {
      url: string;
      kind: string;
      bytes: number;
      contains?: string[];
      lazy?: boolean;
    }[];
  } | null;

OgygiaProfileReport.links

links: {
    html: string;
    json: string;
  };

RunDiff

interface

interface RunDiff {/*…*/}

RunDiff.test

test: string;

RunDiff.kind

kind: OgygiaReport["kind"];

RunDiff.score

score?: {
    base: number;
    head: number;
  };

RunDiff.renderMs

renderMs?: {
    base: number;
    head: number;
  };

RunDiff.js

JS at start, base → head, and the files behind the move (new / grew / shrank / gone, known by what they hold: hashed names change every build)

js?: {
    base: number;
    head: number;
    changes: string[];
  };

RunDiff.findings

findings: {
    added: string[];
    fixed: string[];
  };

RunDiff.hydration

islands that stopped / started hydrating cleanly, per instance (key) — Name#2: clean → failed

hydration?: {
    broke: string[];
    fixed: string[];
  };

RunDiff.session

a recorded session's counts that moved (medians when several runs): errors, unexpected shifts, slow interactions, blocked navigations, its CLS

session?: {
    key: SessionMoveKey;
    base: number;
    head: number;
  }[];

RunDiff.vitals

page vitals that moved past the noise (see CompareOptions)

vitals?: {
    key: string;
    base: number;
    head: number;
  }[];

CompareInput

type

One test's reports from one side: a single run (reports, an ogygia.json body) or several runs of it (runs) — see compare.

type CompareInput = {
  test: string;
  reports: OgygiaReport[];
} | {
  test: string;
  runs: OgygiaReport[][];
};

OgygiaReport

type

type OgygiaReport = OgygiaPageReport | OgygiaHydrationReport | OgygiaSessionReport | OgygiaProfileReport;

OgygiaSessionReport

type

type OgygiaSessionReport = Omit<SessionReport, "findings" | "counts"> & {
  kind: "session";
  engine: OgygiaEngine;
  counts: Omit<SessionReport["counts"], "cls"> & {
    cls?: number;
  };
  findings: (SessionReport["findings"][number] & {
    names: string[];
  })[];
  navigations: {
    url: string;
    t: number | null;
    after: string | null;
    island: string | null;
    approx: boolean;
  }[];
  timeline: unknown[];
  unmeasured: string[];
  server?: (OgygiaProfileReport & {
    error?: undefined;
    window: {
      ms: number;
      requests: {
        method: string;
        path: string;
        route: string | null;
        ms: number;
        status: number | null;
        cpu_ms?: number;
        net_ms?: number | null;
        instance?: string;
      }[];
      instances?: number;
    };
  }) | {
    error: string;
    window?: undefined;
  };
};

OgygiaTestOptions

type

type OgygiaTestOptions = OgygiaOptions & {
  attach?: boolean;
  lean?: boolean;
};

SessionMoveKey

type

type SessionMoveKey = "errors" | "shifts" | "slow" | "navigations" | "cls";