Skip to content
Select language:English

2D widgets and panels in the AR world

Updated 2 min read

AuRea includes a small 2D toolkit in the widgets subpackage. Its purpose is not to build full application interfaces, but to generate 2D graphical content that can be turned into scene objects.

In the current codebase, the transition from widget to 3D scene happens through AuWidget.buildObject(), which creates a textured object on a flat surface.

The flow is:

  1. compose a widget;
  2. draw the widget onto an internal canvas;
  3. convert the canvas into a texture;
  4. apply the texture to a rectangular mesh;
  5. add the resulting object to the world, an anchor, or a group.

The main classes in the subpackage are:

  • AuText: single-line or multiline text;
  • AuImage: a bitmap drawn on the canvas;
  • AuRow: horizontal layout;
  • AuColumn: vertical layout;
  • AuCallout: an information panel with optional title and message;
  • AuInsets: margins around widgets.

These types can be combined to build information panels, labels, callouts, or small spatial overlays.

The widget system is closely related to the graphics subpackage. Types in that package, such as canvas, paint, bitmap, path, and color utilities, provide the low-level drawing tools used to render widget content.

These graphics objects are also useful when creating new custom widgets. In practice, a custom widget is typically defined by deciding how it measures itself and how it draws its content through the drawing primitives exposed by the graphics package.

val paint = AuPaint().apply {
textSize = 32f
color = AuColor.WHITE
}
val widget = AuText(
text = "Hello AR",
paint = paint,
margin = AuInsets.all(16f)
)
val object3d = widget.buildObject(transform)
anchor.add(object3d)

In this example, the widget does not remain a separate 2D element: it immediately becomes a scene object, ready to be treated like any other 3D content.

For richer panels, AuRow and AuColumn can be used to combine text, images, and other widgets. AuCallout already provides a ready-made solution for tooltips, speech bubbles, and callouts attached to markers or points of interest.

A typical composition is:

  • a title at the top;
  • an image and text arranged in a row;
  • final conversion into a 3D panel.

AuWidget.buildObject() also accepts the pixelsPerMeters parameter, which controls the conversion from the widget’s 2D dimensions to the spatial dimensions of the resulting object.

In practical terms:

  • high value: smaller panel in the world;
  • low value: larger panel in the world.

This makes it possible to design the widget in pixels while still controlling its real size in the AR scene.

The panel generated from a widget is also a native object, so it must be treated like any other AuNative. If widgets are used to create temporary objects, those objects should be removed and closed when the panel is no longer needed.

The widget itself, however, is a regular Kotlin object: from a native memory perspective, what requires attention is the final result produced by AuWidget.buildObject().