text: In this short note we introduce ResearchDoom, an implementation of the Doom first-person shooter that can extract detailed metadata from the game. We also introduce the CocoDoom dataset, a collection of pre-recorded data extracted from Doom gaming sessions along with annotations in the MS Coco format. ResearchDoom and CocoDoom can be used to train and evaluate a variety of computer vision methods such as object recognition, detection and segmentation at the level of instances and categories, tracking, ego-motion estimation, monocular depth estimation and scene segmentation. The code and data are available at this http URL
  @cite_5: Computer graphics can not only generate synthetic images and ground truth but it also offers the possibility of constructing virtual worlds in which: (i) an agent can perceive, navigate, and take actions guided by AI algorithms, (ii) properties of the worlds can be modified (e.g., material and reflectance), (iii) physical simulations can be performed, and (iv) algorithms can be learnt and evaluated. But creating realistic virtual worlds is not easy. The game industry, however, has spent a lot of effort creating 3D worlds, which a player can interact with. So researchers can build on these resources to create virtual worlds, provided we can access and modify the internal data structures of the games. To enable this we created an open-source plugin UnrealCV (this http URL) for a popular game engine Unreal Engine 4 (UE4). We show two applications: (i) a proof of concept image dataset, and (ii) linking Caffe with the virtual world to test deep network algorithms.
summary: Another recent effort similar to ResearchDoom is Unreal CV @cite_5 . This engine can also extract metadata similar to ResearchDoom and, by building on the Unreal engine, can potentially be applicable to a huge variety of modern video games. ResearchDoom is much simpler than Unreal CV as it applies to a single and relatively old game. While Doom incorporates several limitations, such as the fact that the camera can only rotate around the vertical axis, the data is nevertheless fairly complex. Furthermore, Doom provides a relatively restricted and consistent world form which data can be extracted for experiments. We leverage the latter fact to define and provide abundant pre-computed data with detailed metadata as well as with annotations using the Microsoft Coco format.