D2Snap

D2Snap is a first-of-its-kind DOM downsampling algorithm, designed for use with LLM-based web agents.

Downsampling is a technique to reduce data through local, lossy consolidation, such that encoded features are largely preserved. A digital image can be locally consolidated by averaging an n-tile of pixels – the depicted object would remain recognisable. We transfer this principle to the DOM: it can analogously be locally consolidated by merging n-subtrees of nodes into a single ’average’ node. What constitutes a feature thereby depends on the intended purpose of the downsampled DOM; in our case, features are implied by the downsampled DOM’s utility as a snapshot for a web agent’s LLM backend. A gradual increase of the downsampling ratio should yield a gradual DOM reduction. Since HTML lexemes have unbounded length, however, monotonic size reduction is a strong aim, and linear size reduction a weak, best-effort aim.

Example

Get an idea of how it works: try the 🧪 Interactive Demo.

<section class="container" tabindex="3" required="true" type="example">
  <div class="mx-auto" data-topic="products" required="false">
    <h1>Our Pizza</h1>
    <div>
      <div class="shadow-lg">
        <h2>Margherita</h2>
        <p>
         A simple classic: mozzarella, tomatoes and basil.
         An everyday choice!
        </p>
        <button type="button">Add</button>
      </div>
      <div class="shadow-lg">
        <h2>Capricciosa</h2>
        <p>
          A rich taste: mozzarella, ham, mushrooms, artichokes and olives.
          A true favourite!
        </p>
        <button type="button">Add</button>
      </div>
    </div>
  </div>
</section>

↓ D2Snap _{(rE = 0.3, rA = 0.3, rT = 0)} ↓

<section class="container" required="true" type="example">
  <div class="mx-auto" required="false">
    # Our Pizza
    <div>
      ## Margherita A simple classic: mozzarella, tomatoes and basil. An everyday choice!
      <button type="button">
        Add
      </button>
      ## Capricciosa A rich taste: mozzarella, ham, mushrooms, artichokes and olives. A true favourite!
      <button type="button">
        Add
      </button>
    </div>
  </div>
</section>

↓ D2Snap _{(rE = 1, rA = 1, rT = 0)} ↓

# Our Pizza ## Margherita A simple classic: mozzarella, tomatoes and basil. An everyday choice!
<button>
  Add
</button>
## Capricciosa A rich taste: mozzarella, ham, mushrooms, artichokes and olives. A true favourite!
<button>
  Add
</button>

Output formatted in debug mode.

Integration

Module

npm install webfuse-com/D2Snap

Install jsdom to use the library with Node.js:

npm install jsdom

import * as D2Snap from "@webfuse-com/d2snap";

Browser

<script src="https://cdn.jsdelivr.net/gh/webfuse-com/D2Snap@main/dist.browser/D2Snap.js"></script>

The browser bundle exposes the API via the global window.D2Snap.

Usage

D2Snap.d2Snap(
  dom: DOM | string,
  rE: number, rA: number, rT: number,
  options?: D2SnapOptions
): Promise<{
  dom: Element;
  innerHTML: string;  // alias: html
  outerHTML: string;
  meta: object;
}>

D2Snap.adaptiveD2Snap(
  dom: DOM | string,
  maxTokens: number = 32768,
  maxIterations: number = 5,
  options?: D2SnapOptions
): Promise<{
  dom: Element;
  innerHTML: string;  // alias: html
  outerHTML: string;
  meta: object;
  parameters: object;
  adaptiveIterations: number;
}>

Read the full API Documentation.

Experiments

Setup

npm install
npm install jsdom

Build

npm run build

Test

npm run test

npm run test:<SUITE> -- [<TEST-NAME=*> # e.g., D2Snap.options.minify] [<FLAG>*]

Suite	Flags	Note
unit
end-to-end
regression
regression:syntax	--next-failure	Over 90 real-world web pages

Evaluate

Provide LLM API provider key(s) to .env (compare example).

cd eval && pip install -r requirements.txt

Success Rate

npm run eval:success:<subject>

<subject> ∈ { gui, dom, bu, D2Snap }

npm run eval:success:D2Snap -- --verbose --split 10,20 --provider openai --model gpt-4o

Does use the LLM backend.

Size Ratio

npm run eval:ratio

Does not use the LLM backend.

Re-create Snapshots

npm run snapshots:create

Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents
_{Thassilo M. Schiepanski   Nicholas Piël
Surfly BV}