unding boxes) and ground text to the visual world. This grounding capability allows for more efficient human-AI interaction for vision-language tasks, as it enables the user to directly point to an object or region in an image, and the model can understand that image region with its spatial locations.

To achieve this, the authors constructed a large-scale dataset of grounded image-text pairs, called GRIT, which was used to train the model. The grounded image-text pairs were built upon a subset of image-tex