Graphics Research - Yenra

The foundations still matter

Rendering, reconstruction, and generation

Rendering starts with a scene description and computes a view. Reconstruction estimates a scene from observations such as photographs. Generation synthesizes content using learned patterns or other rules. Modern systems combine these operations, but their claims should remain distinct: a plausible generated object is not necessarily an accurate reconstruction of something that existed.

What generative AI adds to graphics

Traditional procedural tools can generate content from explicitly designed rules. Learned generators instead capture regularities in training data. They can propose varied appearances and structures from text, reference images, sketches, or other conditions, while introducing new questions about predictability and control.

Diffusion, flow, and compressed representations

Diffusion models learn to reverse a noise-adding process so that sampling can turn noise into structured content. Latent diffusion performs this work in a learned compressed representation rather than directly at full pixel resolution. The latent diffusion paper demonstrated this approach for image synthesis and tasks including inpainting and super-resolution.

Flow-based approaches learn transformations that transport a simple distribution toward the data distribution. Transformers can serve as the network architecture within these systems; “transformer” and “diffusion” describe different aspects of a model. Other generative approaches, including adversarial and autoregressive models, also contribute to graphics research.

Conditioning turns generation into a tool

Text describes a broad intention, but graphics tasks often require more precise inputs: a silhouette, depth map, camera pose, segmentation mask, skeleton, or existing image. Conditioning constrains generation with that information. Researchers aim to improve adherence while preserving visual quality and allowing useful variation.

Inpainting fills or revises a selected region. Outpainting extends an image beyond its original frame. Neither operation guarantees consistency by itself: shadows, reflections, repeated patterns, and object identity still need evaluation.

Neural graphics: from captured views to generated 3D

Neural radiance fields

NeRF, introduced in 2020, represents a scene using a neural network that predicts volume density and view-dependent color. Optimizing against captured images allows new camera views to be rendered. The original method reconstructs a particular scene; it is not a general text-to-3D generator.

3D Gaussian splatting

3D Gaussian Splatting, published in 2023, represents a captured scene with optimized, oriented Gaussian primitives carrying appearance and opacity information. Its renderer demonstrated real-time novel-view synthesis on the paper's evaluated scenes and hardware. Like NeRF, the original method concerns reconstruction rather than prompt-based creation.

These representations can reproduce complex appearance, but a good novel view does not guarantee clean surfaces, correct hidden geometry, or editable material properties. The original captured lighting may be entangled with appearance. Relighting and robust editing require additional methods.

Learning to create new assets

DreamFusion (2022) explored text-to-3D by using a pretrained 2D diffusion model to guide optimization of a 3D representation. This demonstrated how image-generation knowledge could help create objects without training the diffusion model on a labeled 3D dataset. Such guidance still leaves challenges in geometry and consistency across views.

Other research learns 3D generation more directly. TRELLIS, presented at CVPR 2025, uses structured 3D latents and rectified flow transformers, with decoders for representations including meshes, radiance fields, and 3D Gaussians. TRELLIS.2 continues this direction with native, compact structured latents for 3D generation.

NVIDIA's Edify 3D research combines multiview generation and reconstruction to produce meshes, textures, and material maps. These are research examples with different assumptions and outputs, not interchangeable guarantees of a finished production asset.

Animation, video, and interactive worlds

Video generation adds a temporal dimension to image synthesis. Objects must retain identity as they move, become occluded, and reappear. Camera motion should remain coherent with scene structure. Small inconsistencies can accumulate over longer sequences, turning a convincing frame into an unstable shot.

Character animation introduces further constraints: feet should make believable contact with the ground, joints should move appropriately, and motion should transfer to the intended skeleton. A generated video of a walking character is different from a reusable, editable animation rig and motion sequence.

From watching a scene to acting within it

Interactive generation must respond to user actions with low latency while maintaining state. Microsoft Research's Muse (2025) explored a world and human action model for gameplay ideation using data from Bleeding Edge. It offers a concrete example of research connecting generated visuals with actions.

A model that produces plausible continuations is not automatically a general-purpose simulator or game engine. Persistent objects, exact collision rules, controllable outcomes, and reliable behavior outside familiar examples need separate testing. Visual realism alone does not establish physical accuracy.

How generative methods fit a creative workflow

A practical approach combines learned synthesis with explicit scene structure and artist review. Each stage should preserve the controls needed by the next one.

1. Define the target: Specify whether the deliverable is an illustration, a shot, an editable asset, or an interactive scene.
2. Establish structure: Set composition, scale, camera, silhouettes, or a rough layout before exploring appearance.
3. Generate alternatives: Use text and reference inputs to explore materials, shapes, lighting, or motion within those constraints.
4. Inspect beyond the showcase view: Rotate objects, change lighting, examine hidden surfaces, and review full sequences rather than selected frames.
5. Prepare for production: Repair geometry, revise textures, create rigs, or separate layers as the destination requires.
6. Validate in context: Test the asset or sequence in the actual renderer, editor, or engine, including performance and further edits.

AI-assisted denoising and reconstruction can also accelerate parts of rendering without generating the entire scene. Hybrid pipelines let geometry, simulation, and explicit controls handle some requirements while learned models handle others.

What counts as progress?

A research demonstration should be judged against its stated task. Reconstruction, creative generation, and physical simulation require different evidence. A useful evaluation examines both appealing results and systematic failures.

Image-similarity scores help when a reference exists, but can miss structural defects. Text-image alignment does not establish correct geometry. Human preference studies can assess visual appeal, while task-based studies reveal whether creators actually finish work more effectively. No single score captures all of these dimensions.

Open research questions

• Precise editing: How can a model change one requested property while preserving everything else?
• Persistent worlds: How can generated scenes maintain structure and history across long interactions?
• Materials and lighting: How can systems separate reflectance, illumination, and geometry so assets can be relit?
• Efficient representations: How can quality improve without impractical memory, energy, or computation demands?
• Useful training data: How can research improve coverage, documentation, and representation of uncommon structures and appearances?
• Artist control: Which interfaces make generative results understandable, revisable, and dependable?