text: Automatic photo adjustment is to mimic the photo retouching style of professional photographers and automatically adjust photos to the learned style. There have been many attempts to model the tone and the color adjustment globally with low-level color statistics. Also, spatially varying photo adjustment methods have been studied by exploiting high-level features and semantic label maps. Those methods are semantics-aware since the color mapping is dependent on the high-level semantic context. However, their performance is limited to the pre-computed hand-crafted features and it is hard to reflect user's preference to the adjustment. In this paper, we propose a deep neural network that models the semantics-aware photo adjustment. The proposed network exploits bilinear models that are the multiplicative interaction of the color and the contexual features. As the contextual features we propose the semantic adjustment map, which discovers the inherent photo retouching presets that are applied according to the scene context. The proposed method is trained using a robust loss with a scene parsing task. The experimental results show that the proposed method outperforms the existing method both quantitatively and qualitatively. The proposed method also provides users a way to retouch the photo by their own likings by giving customized adjustment maps.
  @cite_9: We present a machine-learned ranking approach for automatically enhancing the color of a photograph. Unlike previous techniques that train on pairs of images before and after adjustment by a human user, our method takes into account the intermediate steps taken in the enhancement process, which provide detailed information on the person's color preferences. To make use of this data, we formulate the color enhancement task as a learning-to-rank problem in which ordered pairs of images are used for training, and then various color enhancements of a novel input image can be evaluated from their corresponding rank values. From the parallels between the decision tree structures we use for ranking and the decisions made by a human during the editing process, we posit that breaking a full enhancement sequence into individual steps can facilitate training. Our experiments show that this approach compares well to existing methods for automatic color enhancement.
  @cite_7: This paper presents methods for personalization of image enhancement, which could be deployed in photo editing software and also in cloud-based image sharing services. We observe that users do have different preferences for enhancing images and that there are groups of people that share similarities in preferences. Our goal is to predict enhancements for novel images belonging to a particular user based on her specific taste, to facilitate the retouching process on large image collections. To that end, we describe an enhancement framework that can learn user preferences in an individual or collaborative way. The proposed system is based on a novel interactive application that allows to collect user's enhancement preferences. We propose algorithms to predict personalized enhancements by learning a preference model from the provided information. Furthermore, the algorithm improves prediction performance as more enhancement examples are progressively added. We conducted experiments via Amazon Mechanical Turk to collect preferences from a large group of people. Results show that the proposed framework can suggest image enhancements more targeted to individual users than commercial tools with global auto-enhancement functionalities.
  @cite_23: Adjusting photographs to obtain compelling renditions requires skill and time. Even contrast and brightness adjustments are challenging because they require taking into account the image content. Photographers are also known for having different retouching preferences. As the result of this complexity, rule-based, one-size-fits-all automatic techniques often fail. This problem can greatly benefit from supervised machine learning but the lack of training data has impeded work in this area. Our first contribution is the creation of a high-quality reference dataset. We collected 5,000 photos, manually annotated them, and hired 5 trained photographers to retouch each picture. The result is a collection of 5 sets of 5,000 example input-output pairs that enable supervised learning. We first use this dataset to predict a user's adjustment from a large training set. We then show that our dataset and features enable the accurate adjustment personalization using a carefully chosen set of training photos. Finally, we introduce difference learning: this method models and predicts difference between users. It frees the user from using predetermined photos for training. We show that difference learning enables accurate prediction using only a handful of examples.
summary: There has been a number of studies for the automatic photo adjustment. Several methods focus on the global tonal adjustment @cite_23 , the color enhancement @cite_9 , and the personalized enhancement @cite_7 . Those methods are global adjustment approaches based on hand-crafted low-level features such as the color histogram, the scene brightness, and the highlight clipping. @cite_7 , Kapoor al proposed a method that discovers the clusters of users that have similar preferences of image enhancement for the personalized adjustment. While the concept of our method may be similar to those methods, the main difference is that we aim to discover the retouching presets that vary according to the local semantics.