UnrealCV

UnrealCV is a project to help computer vision researchers build virtual worlds using Unreal Engine (UE). It extends UE with a plugin by providing:

1. A set of UnrealCV commands to interact with the virtual world.

2. Communication between UE and an external program, such as PyTorch or TensorFlow.

UnrealCV can be used in two ways. The first is using a compiled game binary with UnrealCV embedded. The second is installing the UnrealCV plugin in an Unreal Engine project and using the editor to build a new virtual world.

Please read Tutorial: Getting Started to learn using UnrealCV.

UnrealCV Dev For UnrealZoo

UnrealCV Dev For UnrealZoo contains features in active development and testing. They are currently provided in supported UnrealZoo environments before general-purpose parts are promoted into the open-source plugin. Runtime MCP clients, examples, and an agent skill are published at unrealcv-runtime-mcp; the Unreal Engine C++ Runtime MCP server is not currently open source.

Images generated from the technical demo RealisticRendering

Citation

If you found this project useful, please consider citing our paper

@article{qiu2017unrealcv,
  Author = {Weichao Qiu, Fangwei Zhong, Yi Zhang, Siyuan Qiao,Zihao Xiao, Tae Soo Kim, Yizhou Wang, Alan Yuille},
  Journal = {ACM Multimedia Open Source Software Competition},
  Title = {UnrealCV: Virtual Worlds for Computer Vision},
  Year = {2017}
}

Contact

If you have any suggestion or interested in using UnrealCV, please contact us.