# Introduction To The Idian A Natural Language Platform For Computer Operation

Alan (Xin Rong)
Team: Xin Rong, Chenguang Ma, Lin Lin, Xuan Zhang, Surong Ruan, Xiaochun Yi Dept. of Automation, Tsinghua University alanrong89@gmail.com http://sites.google.com/site/alanrongxin/

## Abstract

The iDian (previously named as the Operation Agent System) is a framework designed to enable computer users to operate software in natural language. Distinct from current speech-recognition systems, our solution supports format-free combinations of orders, and is open to both developers and customers. We used a multi-layer structure to build the entire framework, approached rule-based natural language processing, and implemented demos narrowing down to Windows, text-editing and a few other applications. This essay will firstly give an overview of the entire system, and then scrutinize the functions and structure of the system, and finally discuss the prospective development, esp. on-line interaction functions.

Note: The iDian is designed and implemented in simplified Chinese environment, and the syntactic and semantic models only accept Chinese as input. Any command that is underlined in this document is a translated version, which won"t be recognized by current iDian"s demos in its English form.

user would have to delete them line by line, or type regular expression in the Replace Dialog, like "^p", which is not accessible to all people. Moreover, if the user inputs "I want to make a video call with my son", iDian will firstly ensures the online-communication software to be used, and then searches the contact-list and make a call automatically, which is especially useful for computer beginners or 
"permanent intermediate users".

The iDian provides an entire interaction solution, rather than a front user interface. By providing the special suit, software developers could easily add support for iDian to their programs. Therefore, the iDian builds up a bridge between the manufacturer and the customer of software, boosting up the way people interact with personal computers and associated devices.

## Functions

In order to make the readers get a clearer view on iDian"s functions, here we present several more detailed situations where iDian is used to increase efficiency and help computer beginners.

## Background

Current human-computer interactions rely on graphical user interface. 

The costs and difficulties of GUI designing increases as the software 

![0_image_0.png](0_image_0.png)

Figure 2 Redundant carriage returns in a document copied from a webpage.

back, Alan. Good afternoon! (3) Open the most recent document (4) 
Enter: Confirm ESC: Hide
The iDian is an interface between users and applications, making it possible for users to input actively and get what they need, instead of passively searching lists of options. This way of interaction is much like a command-line interface; nevertheless the entire framework is fundamentally different, providing exciting functions and user experience. For example, if the user types "Delete carriage returns in each line", iDian will make *Word* perform this promptly, otherwise the Example 1: Dealing with formats in *Word*.

Microsoft Word is very "convenient" to work with in most cases. 

However, it could be smarter if more complex orders are accepted. The iDian enables users to manipulate *Word* in format-free expressions which are exactly natural language, so that with only one typing they could complete a task that might have otherwise taken several steps of selections and clicks. Here is a direct example:
A passage copied from a webpage (or PDF document) to *Word* is usually filled with redundant carriage returns (as in Figure 2), which decrease the efficiency when the document is being edited. The solution of most users dealing with similar problems is to delete the carriage returns line by line, which is slow and obviously violates Word"s principles. The "correct" way is: first select the paragraphs, and then press Ctrl+H to call up the Replace Dialog, and then input expressions as below: