Graphics and multimedia - Microsoft Research

Tool

LLMR

This repo contains the code described in LLMR (opens in new tab), implementing the Large Language Model for Mixed Reality framework. This package serves as a prototype for “Speaking the world into existence”, which allows the…

GitHub

Publication

COLE: A Hierarchical Generation Framework for Multi-Layered and Editable Graphic Design

Peidong Jia, Chenxuan Li, Yuhui Yuan, Zeyu Liu, Yichao Shen, Bohan Chen, Xingru Chen, Yinglin Zheng, Dong Chen, Ji Li, Xiaodong Xie, Shanghang Zhang, Baining Guo

November 2023

Microsoft Research Blog

Research Focus: Week of November 22, 2023

November 22, 2023

A new deep-learning compiler for dynamic sparsity; Tongue Tap could make tongue gestures viable for VR/AR headsets; Ranking LLM-Generated Loop Invariants for Program Verification; Assessing the limits of zero-shot foundation models in single-cell biology.

Project

CCEdit

Creative and Controllable AI Video Editing Demo (opens in new tab) Paper (opens in new tab) CCEdit is a comprehensive generative video editing framework meticulously designed to strike a harmonious balance between controllability and creativity…

Publication

TongueTap: Multimodal Tongue Gesture Recognition with Head-Worn Devices

Tan Gemicioglu, R. Michael Winters, Yu-Te Wang, Thomas M. Gable, Ivan Tashev

25th ACM International Conference on Multimodal Interaction | October 2023

Project

Publication

Reality Distortion Room: A Study of User Locomotion Responses to Spatial Augmented Reality Effects

You-Jin Kim, Andrew D. Wilson, Jennifer Jacobs, Tobias Hollerer

International Symposium on Mixed and Augmented Reality (ISMAR) | October 2023

Publication

LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models

Junyi Zhang, Jiaqi Guo, Shizhao Sun, Jian-Guang Lou, Dongmei Zhang

ICCV 2023 | October 2023

Project

Agent AI

Agent-based multimodal AI systems are becoming a ubiquitous presence in our everyday lives. A promising direction for making these systems more interactive is to embody them as agents within specific environments. The grounding of large…

Podcast

Collaborators: Gaming AI with Haiyan Zhang

July 20, 2023

For over a decade, Xbox has been leveraging AI to elevate gaming. Haiyan Zhang, GM of Gaming AI, explores the collaborations behind the work and the potential for generative AI to support better experiences for…

Podcast

Collaborators: Holoportation™ communication technology with Spencer Fowers and Kwame Darko

July 6, 2023

A global team of medical providers is leveraging Holoportation, a Microsoft 3D capture and communication technology, to widen access to specialized care. Computer engineer Spencer Fowers and plastic surgeon Kwame Darko discuss the collaboration.