Dynamic Objects
Overview
Upmade supports the integration of dynamic components in two different ways. You can choose how to embed the content based on what you are building:
- Dynamic objects can contain HTML or framework-specific components while remaining part of the canvas. They can be selected, moved, resized, and saved like other objects.
- The iframe approach hosts HTML, CSS, and JavaScript inside a scoped sandbox environment, keeping generated or user-authored content separate from the main application.
Rendering HTML
The simplest way to use a dynamic object is to build a plain DOM subtree and pass it as element. Upmade mounts it as-is; there is no renderer to register or unregister, and no lifecycle hooks to implement. This is a good fit for static content.
If the native HTML owns runtime behavior, such as event listeners or local calculator state, use a registered renderer even when the content is plain DOM rather than a Vue component. Clipboard copy serializes the object while it is still mounted, so a renderer can return the latest serializable state from onSerialize and recreate fresh DOM listeners from rendererData after paste.
Rendering Components
For richer content, use the registerVueDynamicObjectRenderer helper to register a renderer on UpmadeDynamicObjectRendererRegistry with a stable rendererKey. A regular Vue SFC mounted directly into an engine-managed container may not carry its styles reliably, since style injection differs from a normal app mount — so the helper defines your component as a Vue custom element (defineCustomElement) and mounts that instead, keeping styles encapsulated.
The helper defines the .ce.vue component as a custom element so styles remain encapsulated when the engine mounts it outside the normal Vue component tree. State flows through rendererData:
- On mount, the helper initializes your component from
data(falling back tocreateInitialState()on first mount) and passes it as theinitialStateprop. - The component reports state changes through an
onStateChangeprop. The demo updates the object'srendererDataimmediately for copy/export and the helper returns its latest state fromonUnmount, so remounts and reloads resume from the same value.
Always call the unregister function returned by registerVueDynamicObjectRenderer when the hosting page unmounts.
Dynamic object content follows Upmade's wheel handoff rules: scrollable embedded content consumes wheel input while it can scroll; at a boundary, the wheel is passed back to the canvas viewport.
Rendering iFrames
For user-authored or LLM-authored content, UpmadeIframeObject provides a sandboxed space for HTML, CSS, and JavaScript. The iframe keeps its own content and interaction state while Upmade manages it as a canvas object.
This example uses predefined prompts rather than a live LLM. Each prompt replaces the calculator shown in the iframe, with the loading placeholder drawn over it during the change.
Interacting with iFrame Content
Every UpmadeIframeObject runs a small bridge inside its page. Through it, your application can see which element the user points at or clicks, let the user pick an element, highlight elements, and look elements up by CSS selector. DOM nodes cannot leave the sandboxed page, so elements are reported as plain element descriptions and addressed by CSS selectors.
In this example, the panel shows the element under the pointer and the last clicked and picked element of the card. Pick element turns on picking: the picked element is highlighted and the cursor of an agent moves to it. Highlight buttons outlines every button in the card.
Hover and Click Events
While an iframe object is selected, its page is interactive and the editor emits two events:
objectElementHoverwhenever the element under a mouse or pen pointer changes.elementisnullwhen the pointer leaves the page or the object is deselected.objectElementClickwhen the user clicks an element. The click still reaches the page as usual. A press that moved more than a few pixels, for example to drag the object, is not a click, and clicking a label is reported once.
<script setup lang="ts">
import { ref } from 'vue'
import {
getEditorRef,
UpmadeEditor,
type UpmadeObjectElementClickEvent,
type UpmadeObjectElementHoverEvent,
} from '@upmade/vue-editor'
const editor = getEditorRef('editor')
const hovered = ref<string | null>(null)
function onElementHover(event: CustomEvent<UpmadeObjectElementHoverEvent>) {
hovered.value = event.detail.element?.text ?? null
}
function onElementClick(event: CustomEvent<UpmadeObjectElementClickEvent>) {
const { objectId, element, shiftKey } = event.detail
console.log(objectId, element.selector, shiftKey)
}
</script>
<template>
<UpmadeEditor ref="editor" @objectElementHover="onElementHover" @objectElementClick="onElementClick" />
</template>
Click events include the state of the modifier keys (shiftKey, ctrlKey, altKey, metaKey). Hover events include isPicking.
Picking Elements
Picking lets the user choose an element instead of using it, for example to tell an agent which part of a page to change. Turn it on with setObjectElementPicking(). While picking, the element under the pointer is highlighted, a press or click no longer reaches the page, and the editor emits objectElementPick instead. Disabled buttons and keyboard activation can be picked as well. Pointing at a nested frame inside the page picks the element that contains it.
<script setup lang="ts">
import { getEditorRef, UpmadeEditor, type UpmadeObjectElementPickEvent } from '@upmade/vue-editor'
const editor = getEditorRef('editor')
async function startPicking() {
await editor.value?.selectObjects([iframeObject])
await editor.value?.setObjectElementPicking(iframeObject.id, true, { selector: '[data-upmade-id]' })
}
async function onElementPick(event: CustomEvent<UpmadeObjectElementPickEvent>) {
const { objectId, element } = event.detail
await editor.value?.setObjectElementPicking(objectId, false)
console.log('Picked', element.selector)
}
</script>
<template>
<UpmadeEditor ref="editor" @objectElementPick="onElementPick" />
</template>
| Option | Type | Description |
|---|---|---|
| selector | string | Limits picking to elements that match this CSS selector, for example '[data-upmade-id]'. The closest matching ancestor of the element under the pointer is picked, and content outside any match can't be picked. |
| color | string | Any CSS color for the picking highlight. Defaults to the theme's elementHighlight.color. |
Picking only works while the object is selected, and turning it on does not select the object. It stays on until you turn it off. Pick events include the same modifier keys as click events.
Highlighting Elements
highlightObjectElements() outlines elements inside the page, for example the elements an agent is about to change. Every element that matches a selector is highlighted, up to 100 per selector, and each highlight can have its own color and label. A call replaces the previous highlights of that object:
await editor.value?.highlightObjectElements(iframeObject.id, [
{ selector: 'button' },
{ selector: '[data-upmade-id="checkout"]', color: '#16A34A', label: 'Next step' },
]);
// Remove the highlights of one object, or of all objects
await editor.value?.clearObjectElementHighlights(iframeObject.id);
await editor.value?.clearObjectElementHighlights();
Highlights are drawn inside the page, above its modal dialogs and popovers, and only cover the visible part of each element. They follow the elements when the layout changes or the page scrolls, and they are shown whether or not the object is selected. Without a color, they use the elementHighlight colors of the theme.
Querying Elements
queryObjectElements() describes the elements that match a CSS selector, in document order. Use it to find the element an agent should work on, or to give an LLM an overview of a page:
const buttons = await editor.value?.queryObjectElements(iframeObject.id, 'button');
const named = await editor.value?.queryObjectElements(iframeObject.id, '[data-upmade-id]', 200);
limit defaults to 100 and can be at most 500. The object must be mounted, which it is while it is in or near the viewport. The promise rejects for an invalid selector, and when the page does not answer in time: within 3 seconds, or within 15 seconds while it is still loading.
To point the cursor of an agent at an element with agent.moveTo({ objectId, selector }), see Agents: Driving Agents from Code.
Element Descriptions
Events and queries describe elements as UpmadeObjectElementInfo:
| Property | Type | Description |
|---|---|---|
| selector | string | A CSS selector that matched exactly this element when it was reported. Built from data-upmade-id or id when possible, otherwise from the element's position in the page. |
| upmadeId | string | The element's data-upmade-id attribute. |
| tagName | string | Lowercase tag name, for example 'button'. |
| id | string | The element's id attribute. |
| classList | string[] | The element's classes. |
| text | string | The element's text without script and style content, with collapsed whitespace, truncated to 200 characters. |
| attributes | Record<string, string> | Descriptive attributes: role, aria-*, data-*, name, type, href, src, alt, title, placeholder, and for. Form field values are never included here; see value. |
| state | UpmadeObjectElementState | State the attributes don't show, with only the fields that apply: disabled, checked (true, false, or 'mixed'), selected (options), open (details), focused, editable (text fields, selects, and editable elements), and playing (audio and video). Omitted when there is none. |
| value | string | The current value of a form field, truncated to 500 characters. Only reported by queries with includeValues: true and by interactions, and never for password fields. |
| rect | UpmadeObjectElementRect | The element's bounding box in content coordinates: CSS pixels relative to the top-left corner of the iframe, before canvas zoom, object scale, and rotation. |
| visibleRect | UpmadeObjectElementRect | The visible part of rect, clipped by scrolling containers and the iframe. Omitted when no part of the element can be seen. |
Pass selector back to highlight the element, query it again, or point an agent's cursor at it. A selector built from the element's position, such as main > button:nth-of-type(2), may match another element once the page changes. Add a data-upmade-id to meaningful elements, and ask an LLM that writes pages to do the same, so their selectors keep working when the content is regenerated:
<button data-upmade-id="checkout" type="button">Checkout</button>
Local and Untrusted
Highlights, picking, and the tracking of the elements that agents' cursors stick to are local to the editor. They are not saved, not synchronized to collaborators, and not part of undo and redo. They are kept when the page reloads, for example after setContent(), and when a collaborator updates the object.
The page runs untrusted content, and element descriptions come from the page. Treat their text, attributes, and selectors like any other untrusted input, for example when you pass them to an LLM. The element methods reject invalid selectors, and objects that don't exist or have no content elements. The page also reports hovers, clicks, and picks itself, so untrusted content can report them without a real user action while the object is selected or picking. Don't treat a pick as the user's confirmation of something consequential.
The element features need the page to run scripts. With a sandbox that leaves out allow-scripts, nothing is reported or highlighted and queries reject.
Corners and Shadow
Dynamic objects and iframe objects are drawn as cards: with the theme's corner radius and shadow (see Theming). Set borderRadius or boxShadow on an object to override them for that object, and boxShadow: 'none' to remove its shadow.
Content is clipped to the rounded corners, so it never shows past them. An iframe's document is always clipped. For a dynamic object whose content has to overflow on purpose, such as a dropdown or a popover, set clipContent: false, and round the content's own corners with var(--upmade-content-border-radius), which holds the object's radius.
.my-widget {
border-radius: var(--upmade-content-border-radius);
}
API Details
For the full list of related APIs, see the following reference sections.