Visualize a camera pose frustum from an extrinsic matrix

You have a calibrated camera — a rotation R and a translation t that place it in the world — and you want to see it in the scene: does it look into the bin, how much belt does it cover, is the pose the hand-eye solve just returned plausible? Draw the camera as geometry instead of hunting for a built-in icon. A coordinate frame marks the pose, a wireframe frustum built from the intrinsics shows coverage, and a 4x4 homogeneous matrix places both in world coordinates. This guide works the corner arithmetic with real numbers, lists the drawing steps in Python, shows the per-frame update that avoids the empty-screen trap, and gives three checks that catch a wrong matrix before it costs a robot pick.

What geometry makes a camera visible in a 3D visualizer?

Draw two geometries and add both once: a coordinate frame whose origin marks the camera pose, and a wireframe pyramid with five points and eight edges whose apex sits at that same origin and whose base spans the four image corners at your chosen frustum depth.

The frame gives you orientation at a glance: red, green and blue axes mean x, y and z, so a rolled or upside-down camera is obvious from across the room. Build it at unit size with the mesh factory, open3d.geometry.TriangleMesh.create_coordinate_frame(size=1.0, origin=[0, 0, 0]).

The pyramid is the viewing frustum, the region of space that can appear on the screen. Its apex is the camera centre; its base is the image rectangle pushed d metres down the view axis. You choose d, and every corner scales linearly with it. Everything below assumes the pinhole convention your intrinsics are written in: x right, y down, z forward.

How do you compute the four frustum corners from the intrinsics?

Each image corner (u, v) becomes a ray direction ((u - cx)/fx, (v - cy)/fy, 1), so with fx = 800, cx = 640 and a 1280×720 image the four corners are (±0.8, ±0.45, 1.0) at d = 1.0 m — a 77.3° × 48.5° field of view.

The half-angles confirm the shape: atan(0.8) = 38.7° per side gives 77.3° horizontal, and atan(0.45) = 24.2° per side gives 48.5° vertical. If fx and fy differ the base is simply a rectangle and the same formula holds per axis. Calibrated intrinsics already have distortion removed, so the straight pyramid matches what the camera sees.

How does the extrinsic matrix place the frustum in the world?

Multiply, never add: stack R and t into the 4x4 homogeneous matrix E, homogenize each camera-frame point with a trailing 1, and compute E · p — with R = diag(1, -1, -1) and t = (0, 0, 1.5 m), the corner (-0.8, -0.45, 1.0) lands at (-0.8, 0.45, 0.5).

The example pose used throughout this guide: the camera hangs 1.5 m above the world origin looking straight down, so R = diag(1, -1, -1) — x kept, y and z flipped — and t = (0, 0, 1.5) m. Corner 1 in camera coordinates is (-0.8, -0.45, 1.0), so:

• R · p = (-0.8, 0.45, -1.0)
• p + t = (-0.8, 0.45, 0.5)

The apex (0, 0, 0) lands at (0, 0, 1.5). The base centre (0, 0, 1.0) lands at (0, 0, 0.5), exactly 1.0 m down the view axis, with the base spanning 1.6 m × 0.9 m. At the floor, 1.5 m down the ray, coverage would be 1.6 × 1.5 = 2.4 m wide and 0.9 × 1.5 = 1.35 m deep, because footprint grows linearly with distance.

Convert whatever your SDK hands you into R and t first: a hand-eye quaternion or a rotation vector plus its translation. A hand-eye solve returns world-from-camera or its inverse — the AX=XB calibration write-up walks through which one your pose chain produces — and the inverse (Rᵀ, -Rᵀ · t) flips one into the other. Capture the poses for the solve with the hand-eye pose capture workflow.

What does the drawing code look like, step by step?

Seven steps: define the intrinsics, build the five camera-frame points and eight edge pairs, wrap them in a line-geometry object, build a unit coordinate frame, apply E to both, then create the window and add both geometries once before the loop starts.

1. Write the intrinsics as float64: W = 1280, H = 720, fx = fy = 800.0 px, cx = 640.0 px, cy = 360.0 px, d = 1.0 m.
2. Build the four corner rays from the pixel corners, scale by d, and stack the apex on top: five points.
3. List the eight edge index pairs — four apex spokes (0-1 through 0-4) and four base sides (1-2, 2-3, 3-4, 4-1).
4. Assemble E from R and t.
5. Map the points to world: pts_w = (E[:3, :3] @ pts.T).T + E[:3, 3].
6. Wrap the world points and edge pairs in a line-geometry object; build the unit frame at the origin and apply E to it as well.
7. Create the window and add both geometries once.

import numpy as np
import open3d

# Illustrative values: intrinsics in pixels, depth in metres
W, H = 1280, 720
fx = fy = 800.0
cx, cy = 640.0, 360.0
d = 1.0

# Five camera-frame points: apex + four image corners at depth d
uv = np.array([[0, 0], [W, 0], [W, H], [0, H]], dtype=np.float64)
rays = np.column_stack([(uv[:, 0] - cx) / fx,
                        (uv[:, 1] - cy) / fy,
                        np.ones(4)])
pts = np.vstack([np.zeros((1, 3)), d * rays])      # (5, 3) float64

# Eight edges: four apex spokes, four base sides
edges = np.array([[0, 1], [0, 2], [0, 3], [0, 4],
                  [1, 2], [2, 3], [3, 4], [4, 1]], dtype=np.int32)

# World-from-camera extrinsic, 4x4 homogeneous (illustrative pose)
E = np.eye(4)
E[:3, :3] = np.array([[1.0, 0.0, 0.0],
                      [0.0, -1.0, 0.0],
                      [0.0, 0.0, -1.0]])
E[:3, 3] = [0.0, 0.0, 1.5]

pts_w = (E[:3, :3] @ pts.T).T + E[:3, 3]           # (5, 3) world

# Wireframe from points + index pairs; pose frame moved by the same E
lines = open3d.geometry.LineSet(open3d.utility.Vector3dVector(pts_w),
                                open3d.utility.Vector2iVector(edges))
frame = open3d.geometry.TriangleMesh.create_coordinate_frame(size=1.0, origin=[0, 0, 0])
frame.transform(E)

vis = open3d.visualization.Visualizer()
vis.create_window(width=W, height=H)
vis.add_geometry(frame)
vis.add_geometry(lines)

How do you update the pose every loop without an empty screen?

Add the geometries once, then per loop rewrite the points of the same line geometry in place, call the visualizer's update call for that geometry, pump the window's events and run update_render(); re-adding a fresh object each frame is what empties the screen.

The cause is worth naming. add_geometry() registers an object with the render list; call it with a new object every frame and the list is rebuilt, the old frustum is dropped, the new one never gets a paint cycle, and the window goes blank at the next update_render(). Add once at set-up and keep the object handles.

Per frame the camera-frame points never change, only E does, so the update is one 5×3 matmul and a rewrite:

while True:
    E = current_extrinsic()              # your pose source, 4x4 float64
    pts_w = (E[:3, :3] @ pts.T).T + E[:3, 3]
    lines.points = open3d.utility.Vector3dVector(pts_w)  # rewrite in place
    vis.update_geometry(lines)           # mark the registered object changed
    vis.update_render()
    vis.poll_events()                    # pump UI events or the window never repaints

For the axes frame, apply the delta transform E_new @ np.linalg.inv(E_prev) each loop, or leave it static while you tune. The same rewrite-in-place pattern drives a moving pose stream, as in trajectory visualization from pose updates.

How do you verify the frustum is actually right?

Three checks decide it: the corner rays must reproduce the measured field of view, the base centre must sit exactly d metres from the apex along the view axis, and a world point you have tape-measured and image-projected must land inside the pyramid.

1. Field of view. atan(640 / 800) = 38.7° per side, so at 1.0 m the base must span 2 × 0.8 = 1.6 m. Lay a rule on the belt at a tape-measured 1.0 m from the lens; the pyramid's base edges should land on the 1.6 m marks.
2. Apex-to-base distance. Base centre minus apex must have length exactly d: (0, 0, 0.5) − (0, 0, 1.5) gives 1.0 m in the worked example. Any other value means d was applied twice or the points were never homogenized.
3. Projection consistency. Take a world point you measured, map it into camera coordinates with the inverse of E, project it with u = fx · (x / z) + cx, and check the pixel you actually see in the live image; that same point must sit inside the drawn pyramid. If the pyramid faces away from the scene, you have a convention mismatch, not a bad calibration.

Chain robot and camera transforms by multiplication, never addition — the pose-composition rule for robot poses holds regardless of controller — and the eye-to-hand object pose transform is the full static-camera chain.

Which cameras does this workflow suit best?

It suits any camera whose calibration hands you R and t — most often the cell's 3D camera, a laser profiler over a conveyor or a projected structured-light camera over a bin, because a pose bug there shows up as picks off by tens of millimetres.

It earns its keep in three spots: after a hand-eye capture, to eyeball the result before the robot moves; during commissioning of eye-to-hand cells, where a transposed rotation shows up as a frustum facing the wall; and after a lens or camera swap, to see the new coverage before cutting a bracket.

On the shop floor the pose you draw usually belongs to the cell's 3D sensor. Two from the 3D sensors range; the camera finder filters the full list by type:

Pick the parts, draw the frustum at the calibrated pose, and only then jog the robot. If your matrix and your scene disagree, send what you have through the contact page and we will read it with you; the 3D topic page collects the rest of the pose and transform guides.