\section{Introduction}\label{sec:intro}
In the recent years, there has been a tremendous increase in the amount of digital multimedia data, especially the video content, both in the form of video archives and live streams. According to statistics~\cite{burgess2018youtube}, 300 hours of video is being uploaded every minute on the YouTube. A key factor responsible for this enormous increase is the availability of low-cost smart phones equipped with cameras. With such huge collections of data, there is a need to have efficient as well as effective retrieval techniques allowing users retrieve the desired content. Traditionally, videos are mostly stored with user assigned annotations or keywords which are called tags. When a content is to be searched, a keyword provided as query is matched with these tags to retrieve the relevant content. The assigned tags, naturally, cannot encompass the rich video content leading to a constrained retrieval. A better and more effective strategy is to search within the actual content rather than simply matching the tags i.e. Content based Image or Video Retrieval. CBVR systems have been researched and developed for long a time and allow a smarter way of retrieving the desired content. The term content may refer to the visual content (for example objects or persons in the video), audio content (the spoken keywords for instance) or the textual content (News tickers, anchor names, score cards etc.). Among these, the focus of our current study lies on textual content. More specifically, we target a smart retrieval system that exploits the textual content in videos as an index. \\

The textual content in video can be categorized into two broad classes, scene text and caption text. Scene text (Figure~\ref{fig:scenetext}) is captured through camera during the video recording process and may not always be correlated with the content. Examples of scene text include advertisement banners, sign boards, text on a T-shirt etc. Scene text is commonly employed for applications like robot navigation and assistance systems for the visually impaired. Artificial or caption text (Figure~\ref{fig:captext}) is superimposed on video and typical examples include News tickers, movie credits, score cards, names of anchors etc. Caption text is generally correlated with the video content and is mostly applied for semantic retrieval of videos. \\
\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure1.pdf}
	\caption{Examples of Scene Text}
		\label{fig:scenetext}
\end{figure}

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure2.pdf}
	\caption{Examples of Caption Text}
		\label{fig:captext}
\end{figure}

The key components of a textual content based indexing and retrieval system include detection of text regions~\cite{mirza2018urdu}, extraction of text (segmentation from background)~\cite{ali2019}, identification of script (for multi-script videos)~\cite{ali2016} and finally recognition of text (through a video OCR)~\cite{hayat2018ligature}. Among these, we focus on detection of text in the current study. Detection of text can be carried out using unsupervised~\cite{baran2018automated, dai2018scene, banerjee2013robust}, supervised~\cite{xu2018end, he2018single, liao2017textboxes} or hybrid~\cite{mirza2018urdu, shrivastava2017learning} approaches. Unsupervised text detection employs image analysis techniques to discriminate between text and non-text regions. Supervised methods, on the other hand, involve training a learning algorithm with examples of text and non-text regions to discriminate between the two. In some cases, a combination of the two techniques is employed where the candidate text regions identified by unsupervised methods are validated through a supervised approach. \\

This paper presents a comprehensive framework for video text detection in a multi-script environment. Though we primarily target cursive caption text, since video frames frequently contain text in more than one script, text in the Roman script is also detected by the proposed technique. The key highlights of this study are outlined in the following.

\begin{itemize}
    \item Development of a comprehensive dataset of video images with ground truth information supporting evaluation of detection and recognition tasks.
    \item Adaptation of various deep learning based object detectors for detection of textual content.
    \item Combination of text detection and script identification in a single end-to-end system.
    \item Validation of proposed technique through an extensive series of experiments and a comprehensive performance comparison of various detectors.
\end{itemize}

The paper is organized as follows. In the next section, we present an overview of the current state-of-the-art on detection of textual content in videos. In Section~\ref{sec:Dataset}, we introduce the dataset developed in our study along with the ground truth information. Section~\ref{sec:Methodology} presents the details of the proposed framework while Section~\ref{sec:Experiments} presents the experimental protocol, the realized results and the corresponding discussion. Finally, Section~\ref{sec:Conclusion} concludes the paper with a discussion on open challenges on this subject.

\section{Background}\label{sec:lit}
Detection of textual content in videos, images, documents and natural scenes has been investigated for more than four decades. The domain has matured progressively over the years starting with trivial image analysis based systems to complex end-to-end learning based systems. We discuss notable contributions to text detection in the following while detailed surveys on the problem (and related problems) can be found in~\cite{Ye2015,wang2016,yin2016text,zhang2013,sharma2012recent}. \\

Text detection refers to localization of textual content in images. Techniques proposed for detection of text are typically categorized into unsupervised and supervised approaches. While unsupervised approaches primarily rely on image analysis techniques to segment text from background, supervised methods involve training a learning algorithm to discriminate between text and non-text regions.\\

Unsupervised text detection techniques include edge-based methods~\cite{baran2018automated,dai2018scene,banerjee2013robust,huang2013scene,Jamil:2011} which (assume and) exploit the high contrast between text and its background; connected component based methods~\cite{Kiran:2012,koo2013scene,lee2010scene,pan2011hybrid} which mostly rely on the color/intensity of text pixels and texture-based methods~\cite{XiangBai2016,Huang:2011} which consider textual content in the image as a unique texture that distinguishes itself from the non-text regions. Texture based methods have remained a popular choice of researchers and features based on Gabor filters~\cite{gabor1946}, wavelets~\cite{ye2005fast}, curvelets~\cite{joutel2007curvelets}, local binary patterns (LBP)~\cite{anthimopoulos2010two}, discrete cosine transformation (DCT)~\cite{zhong2000automatic}, histograms of oriented gradients (HoG)~\cite{dalal2005histograms} and Fourier transformation~\cite{shivakumara2010new} have been investigated in the literature. Another common category of techniques includes color-based methods~\cite{yi2012localizing,Yi:2007,Shivakumara:2010} which are similar in many aspects to the component-based methods and employ color information of text pixels to distinguish it from non-text regions. \\

Supervised approaches for detection of textual content typically employ state-of-the-art learning algorithms which are trained on examples of text and non-text blocks either using pixel values or by first extracting relevant features. Classifiers like naive Bayes~\cite{shivakumara2012multioriented}, Support Vector Machine~\cite{Zhen:2009}, Artificial Neural Network~\cite{Yin:2013,mirza2018urdu} and Deep Neural Networks~\cite{lecun2015deep} have been investigated for this problem over the years. \\

In the recent years, deep learning based solutions have been widely applied to a variety of recognition problems and have outperformed the traditional techniques. Among deep learning based techniques adapted for text detection, Huang et al.~\cite{huang2014robust} employed sliding windows with CNNs to detect textual regions in low resolution scene images. Likewise, fully convolutional networks are explored for detection of textaul regions in~\cite{zhang2016multi} and the technique is evaluated on various ICDAR datasets. A similar work is presented by Gupta et al.~\cite{gupta2016synthetic} where CNNs are trained using synthetic data for detection of text at multiple scales from natural images. Another method called `SegLink', is proposed in~\cite{shi2017detecting} that relies on decomposing the text into segments (oriented boxes of words or lines) and links (connecting two adjacent segments). The segments and links are detected using fully convolutional networks at multiple scales and combined together to detect the complete text line. In~\cite{tian2016detecting} vertical anchor based method is reported that predicts text and non-text scores of fixed size regions and reports high detection performance on the ICDAR 2013 and ICDAR 2015 datasets. In another notable work, Wang et al.~\cite{wang2018crf} present a framework based on conditional random field (CRF) to detect text in scene images. Authors define a cost function by considering the color, stroke, shape and spatial features with CNN for effective detection of textual regions.\\

Among other end-to-end trainable deep neural networks based systems, Liao et al.~\cite{liao2017textboxes} present a system called `TextBoxes' which detects text in natural images in a single forward pass network. The technique was later extended to `TextBoxes++' and was evaluated on four public databases outperforming the state-of-the-art. He et al.~\cite{he2018single} improved the convolutional layer of CNNs to detect text with arbitrary orientation. EAST~\cite{zhou2017east}, is another well-known scene text detector that provides promising results in challenging scenarios. In another study~\cite{xu2018end}, an ensemble of Convolutional Neural Networks (CNNs) is trained on synthetic data to detect video text in East Asian languages.\\

The literature is relatively limited once it comes to detection of cursive caption text. Among one of the preliminary works, Jamil et al.~\cite{jamil2011edge} exploit edge based features with morphological processing to detect Urdu caption text from a small set of 150 video frames. The same study was extended by Raza et al.~\cite{siddiqi2012database} and evaluated on a larger set of 1000 video frames reporting a recall of 0.80. The dataset of images termed as IPC Artificial Text dataset~\cite{siddiqi2012database} was also made publicly available. In a later study~\cite{raza2013multilingual} by the same group, the authors proposed a cascaded framework of spatial transforms to detect caption text in five different scripts including Arabic and Urdu. In a relatively recent work on detection of Arabic caption text, Zayene et al.~\cite{Oussama2016} employ a combination of stroke width transform with a convolutional auto-encoder and evaluate the technique on a publicly available dataset AcTiV-DB~\cite{Oussama2015}. In one of our previous works~\cite{mirza2018urdu}, we investigated a combination of image analysis techniques with textural features to detect textual regions in video frames and realized an F-measure of 0.80 on 1000 images.\\

Summarizing, it can be concluded that the problem of text detection has been dominated by the application of different deep learning based techniques in the recent years. The availability of benchmark datasets has also contributed to the rapid developments in this area. While detection of text in languages based on the Latin alphabet has received significant research attention and is very much mature, detection of cursive text still remains a relatively less addressed and challenging issue. Development of a (generic) text detector that could work in multi-script environments also remains an open problem.\\

In the next section, we introduce the dataset that has been collected and labeled as a part of this study.

\section{Dataset}\label{sec:Dataset}
Availability of labeled datasets is of utmost importance for algorithmic development and evaluation of any computerized system. From the perspective of Urdu caption text detection, a dataset of 1000 labeled video frames has been made publicly available~\cite{siddiqi2012database}. A collection of 1000 frames, however, seems to be very small to generalize the findings for practical applications. We, therefore, collected and labeled a comprehensive dataset of video frames allowing evaluation of text detection and text recognition tasks. We collected a set of 46 videos from four different News channels in Pakistan. All videos are recorded at a resolution of $900 \times 600$ and a frame rate of 25 fps. Frames in these video contain textual content in two languages, (cursive) Urdu and English. The collected video frames are labeled from two perspectives, detection and recognition. For detection, the bounding rectangle of all text regions in a frame is labeled and stored. Similarly, for recognition, the transcription of each text line is stored as ground truth.

In the literature, several evaluation metrics have been proposed to evaluate the performance of text detection systems~\cite{jamil2011edge,lucas2005icdar,wolf2006object}.  In our system, for evaluation of the text detection module, we employ the most commonly used area based precision and recall measures reported in~\cite{jamil2011edge} and defined as follows.\\

Let $A_E$ be the estimated text area given by the system and $A_T$ be the ground truth text area, then the precision $P$ and recall $R$ are defined as:
\begin{equation}
	P~ =~\frac{~A_E~ \cap~ A_T}{A_E}
\end{equation}

\begin{equation}
	R~ =~ \frac{~A_E~ \cap~ A_T}{A_T}
\end{equation}
The precision and recall measures can be combined in a single F-measure as follows.

\begin{equation}
	F~=~ \frac{~2~ \times ~Precision~ \times~ Recall~ }{~Precision ~+~ Recall}
\end{equation}

The same idea can be extended to multiple images by simply summing up area of intersection and dividing by the total ground truth area (in $N$ images) for recall and the total detected area for precision. To compute these measures, for each frame, we need to store the actual location of the textual content. The text detected automatically by the system can then be compared with the ground truth text regions to compute precision, recall and F-measure. The idea is illustrated in Figure~\ref{fig:textreg}. Figure~\ref{fig:textreg}-a illustrates an example where the text regions detected by the system are shown while Figure~\ref{fig:textreg}-b illustrates the ground truth text locations for the given frame. The detected and ground truth text regions can be compared to compute the metrics defined earlier and quantify the detection performance.\\

\begin{figure*}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure3.pdf}
	\caption{Text regions in an image and the corresponding ground truth image}
	\label{fig:textreg}
\end{figure*}

To facilitate the labeling process and standardize the ground truth data, a comprehensive labeling tool has been developed that allows storing the location of each textual region in a frame along with its ground truth transcription. The location is stored in terms of the $x$ and $y$ coordinates of the top left of the bounding box along its $width$ and $height$. The ground truth information of each frame is stored as an XML file that comprises frame meta data and the information on textual regions. A screen shot of the labeling tool is presented in Figure~\ref{fig:GTTool} while the ground truth information of an example frame is illustrated in Figure~\ref{fig:XMLScreen}.\\
\begin{figure*}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure4.pdf}
		\caption{Screen shot of ground truth labeling tool for text data}
	\label{fig:GTTool}
\end{figure*}

\begin{figure*}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure5.pdf}
	\caption{An XML file containing ground truth information of a frame}
		\label{fig:XMLScreen}
\end{figure*}

It is known that videos typically contain 25--30 frames per second; consequently, successive frames in a video contain redundant information (both visual and textual content).  From the view point of automatic analysis systems, frames with unique content are of interest. Hence, each single video frame does not need to be labeled as major proportions of such frames will have exactly the same textual information. In our study, we have extracted more than 11,000 frames from videos with an attempt to have as much unique text as possible. The statistics of videos, frames and text lines of our dataset are presented in Table~\ref{tab:channelStats}.
Inspired from the Arabic caption text dataset AcTiV-DB~\cite{Oussama2015}, we have named the our dataset UTiV (Urdu Text in Video). The dataset along with its ground truth has also been made publicly available\footnote{http://cbvir.media-tics.net/} to support quantitative evaluation of text detection and recognition tasks.\\

\begin{table*}[!ht]
\centering
\caption{Statistics of labeled video frames}
\label{tab:channelStats}
\begin{tabular}{llcccc}
\hline
\textbf{S\#} & \textbf{Channel} & \textbf{Videos} & \textbf{Labeled Images} & \textbf{Urdu Lines} & \textbf{English Lines}\\
\hline
1 & Ary News & 7 & 3,206 & 10,250 & 3,605\\
2 & Samaa News & 13 & 2,503 & 10,961 & 4,411\\
3 & Dunya News & 16 & 3,059 & 10,723 & 8,861\\
4 & Express News & 10 & 2,424 & 8,536 & 6,755\\
& \textbf{Total} & \textbf{46}& \textbf{11,203} &  \textbf{40,470} & \textbf{23,632}\\
\hline
\end{tabular}
\end{table*}

\section{Methods}\label{sec:Methodology}
This section presents the details of detecting textual content from video frames. Detection relies on adapting object detectors based on deep convolutional neural networks for text regions. Once the text is detected, script of the detected text is identified by employing the ConvNets in a classification framework. Subsequently, text detection and script identification are combined in a single end-to-end system that detects the textual content along with its script. Details are presented in the following sections.

\subsection{Deep Learning based Object Detectors}
Deep neural networks enjoy a renewed interest of the machine learning community thanks primarily to the availability of high performance computing hardware (GPUs) as well as large data sets to train these systems. A major development contributing to the current fame of deep learning was the application of ConvNets by Krizhevsky et al.~\cite{krizhevsky2012imagenet} on the ImageNet Large Scale Visual Recognition competition~\cite{russakovsky2015imagenet}, which greatly reduced the error rates. Since then, CNNs are considered to be state-of-the-art feature extractors and classifiers~\cite{simonyan2014very, szegedy2015going} and have been applied to a variety of recognition tasks~\cite{bouchain2006character, uijlings2013selective, farfade2015multi}. \\

While traditional CNNs are typically employed for object classification, Region-based Convolutional Networks (R-CNN)~\cite{girshick2014rich} and their further enhancements Fast R-CNN~\cite{girshick2015fast} and Faster R-CNN~\cite{ren2015faster} adapt CNNs for object detection. In addition to different variants of R-CNN, a number of new architectures have also been proposed in the recent years for real time object detection. The most notable of these include YOLO (You Only Look Once)~\cite{redmon2016you} and SSD (Single Shot Detector)~\cite{liu2016ssd}. Each of these object detectors can be trained to detect $C$ object classes (plus one for the background). The output of the detector is the location of the bounding box (four coordinates) containing one of the $C$ classes as well as the class confidence score.\\

In our study, for detection of textual content in a given frame, we investigated a number of CNN based object detectors. Although, many object detectors are trained with thousands of class examples and provide high accuracy in detection and recognition of different objects, these object detectors can not be directly applied to identify text regions in images. These models have to be tuned to the specific problem of discrimination of text from non-text regions. The convolutional base of these models can be trained from scratch or, known pre-trained models can be fine-tuned by training them on text and non-text regions. In our study, we investigated the following object detectors for localization of text regions.

\begin{itemize}
	\item Faster R-CNN
	\item Region-Based Fully Convolutional Networks (R-FCN)
	\item Single Shot Detector (SSD)
	\item You Only Look Once (YOLO)
\end{itemize}

For completeness, we provide a brief overview of these object detectors in the following sections.

\subsubsection{Faster R-CNN}
Faster R-CNN~\cite{ren2015faster} is an enhanced version of its predecessors R-CNN~\cite{girshick2014rich} and Fast R-CNN~\cite{girshick2015fast}. Each of these detectors exploits the powerful features of ConvNets for object localization as well as classification. R-CNN was one of the first attempts to apply ConvNets for object detection. An R-CNN scans the input image for potential objects using Selective Search~\cite{uijlings2013selective} that generates around 2,000 region proposals. Each of these region proposals is then fed to a CNN for feature extraction. The output of the CNN is finally employed by an SVM to classify the object and a linear regressor to tighten the bounding box. R-CNN was enhanced in terms of training efficiency by extending it to Fast R-CNN~\cite{girshick2015fast}. In Fast R-CNN, rather than separately feeding each region proposal to the ConvNet, convolution is performed only once on the complete image and the region proposals are projected on the feature maps. Furthermore, the SVM in R-CNN was replaced by a softmax layer extending the network to predict the class labels rather than using a separate model. While Fast R-CNN significantly reduced the time complexity of the basic R-CNN, a major bottleneck was the selective search algorithm to generate the region proposals. This was addressed through Region Proposal Network (RPN) in Faster R-CNN~\cite{ren2015faster} which shares convolutional features with the detection network. RPN predicts region proposals which are then fed to the detection network to identify the object class and refine the bounding boxes produced by the RPN. A summary of various R-CNN models in presented in Figure~\ref{fig:frcnn}.

\begin{figure*}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure6.pdf}
	\caption{Summary of R-CNN Family based Object Detectors}
\label{fig:frcnn}

\end{figure*}

\subsubsection{You Only Look Once (YOLO)}
YOLO~\cite{redmon2016you} takes a different approach to object detection primarily focusing on improving the detection speed (rather than accuracy). As the name suggests, YOLO employs a single pass of the convolutional network for localization and classification of objects from the the input images. The input image is divided into a grid and an object is expected to be detected by the grid which holds the center of the object. Each cell in the grid predicts up to two bounding boxes (and class probabilities). The network comprises 24 convolutional and  fully connected layers. YOLO works in real time but in terms of accuracy, it is known to make significant localization errors in comparison to region based object detectors (Faster R-CNN for instance).
YOLO was later enhanced to YOLO9000~\cite{redmon2017yolo9000} by introducing batch normalization, increasing the resolution of the input image (by a factor of 2) and introducing the concept of anchor boxes. YOLO9000 employs Darknet 19 architecture with 19 convolutional layers, 5 max pooling layers and a softmax layer for classification objects. Incremental improvements in YOLO v2 resulted in YOLO v3~\cite{redmon2018yolov3} that uses logistic regerssion to predict the score of objectness for each bounding box. Furthermore, it employs class-wise logistic classifiers (rather than softmax) allowing multi-label classification.

\subsubsection{Single Shot Detector (SSD)}
Unlike the R-CNN series object detectors which require two shots to detect objects in an image, Single Shot Multi-box Detector~\cite{liu2016ssd}, as the name suggests, requires a single shot to detect objects (similar to YOLO). SSD relies on the idea of default boxes and multi-scale predictions and directly applies bounding box regression to the default boxes without generating the region proposals. Detection at multiple scales are handled by exploiting the feature maps of different convolutional layers corresponding to different receptive fields in the input image. The architecture has an input size of $300 \times 300 \times 3$ and primarily builds on the VGG-16 architecture discarding the fully connected layers. VGG-16 is used as base network mainly due to its robust performance of image classification tasks. The bounding box regression technique of SSD is inspired by~\cite{szegedy2015going} while the MultiBox relies on priors, the pre-computed fixed size bounding boxes. The priors are selected in such a way that their Intersection over Union ratio (with ground truth objects) is greater than 0.5. The MultiBox starts with the priors as predictions and attempt to regress closer to the ground truth bounding boxes. SSD works in real time but requires images of fixed square size and is known to miss small objects in the image.

\subsubsection{Region-Based Fully Convolutional Networks (R-FCN) }
R-FCN~\cite{dai2016r} builds on the idea of increasing the detection accuracy by maximizing the shared calculations. R-FCN generates position-sensitive score maps to represent different relative positions of an object. An object is represented by $k^2$ relative positions dividing it into a grid of size $k \times k$. A ConvNet (ResNet in the original R-FCN paper) sweeps the input image and an additional fully convoltional layer produces the position-sensitive scores in $k^2 \times (C+1)$ score maps where $C$ is the number of classes plus 1 class for the background. A fully convolutional proposal network generates regions of interest which are divided in $k^2$ bins and the corresponding class probabilities are obtained from the score maps. The scores are averaged to convert the $k^2 \times (C+1)$ values into a one dimensional $(C+1)$ sized vector which is finally fed to the softmax layer for classification. Localization is carried out using the bounding box regression similar to other object detectors. R-FCN speeds up the detection in comparison to Faster R-CNN but compared to other Single Shot methods, it requires more computational resources.

\subsection{Adapting Object Detectors for Text Detection}
In the context of object detection, the problem of text detection can be formulated as a two class problem. The text regions represent object of interest while the non-text regions need to be ignored. The object detectors discussed in the previous sections are adapted for text detection using two pre-trained models, ResNet 101~\cite{he2016deep} and Inception v2~\cite{ioffe2015batch}. These models are trained on the large scale Microsoft COCO (Common Objects in Context) database~\cite{lin2014microsoft}. The database contains images of 91 different object types with a total of 2.5 million labeled instances in 328K images. The pre-trained network serves as starting point rather than random weight initialization and the network is made to learn the specific class labels (text or non-text) by continuing back propagation. The ground truth localization information of the textual regions in the video frames is employed for training the models, the overall workflow being illustrated in Figure~\ref{fig:traintestmethod}. \\

A critical aspect in employing object detectors for text detection is the choice of anchor boxes. The anchor boxes in all the detectors have been designed to detect general object categories. Text appearing in videos has specific geometric properties in terms of size and aspect ratio hence the default anchor boxes of the detectors need to be adapted to detect textual regions. We carried out a comprehensive analysis of the textual regions in terms of width, height and aspect ratios of the bounding boxes. As a result of this analysis we have chosen a base anchor of size $256 \times 256$. To each anchor box we apply three scales ($1.0, 2.0, 5.0$) and five aspect ratios ($0.125, 0.1875, 0.25, 0.375, 0.50$) as illustrated in Figure~\ref{fig:Scales}. Models are fine-tuned using the proposed anchor boxes and the effectiveness of these anchor boxes is validated through experimental study as presented in Section~\ref{sec:Experiments} of the paper.   \\

\begin{figure*}[!ht]
	\centering
		\includegraphics[width=1.0\textwidth]{figure7.pdf}
	\caption{Overview of adapting object detectors for text detection}
\label{fig:traintestmethod}
\end{figure*}

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.75\textwidth]{figure8.pdf}
	\caption{Anchor boxes (base size $256 \times 256$) at three scales ($1.0, 2.0, 5.0$) and five aspect ratios ($0.125, 0.1875, 0.25, 0.375, 0.50$)}
\label{fig:Scales}
\end{figure}

\subsection{Script Identification}
As discussed earlier, we primarily target detection of cursive caption text. However, like many practical scenarios, video frames in our case contain bilingual textual content (Urdu \& English). Consequently, once the text is detected, we need to identify the script of each detected region (Figure~\ref{fig:scriptIden}) so that the subsequent processing of each type of script can be carried out by the respective recognition engine. For script identification, we employ CNNs in a classification framework (rather than detection). Urdu and English text lines are employed to fine-tune CNNs to discriminate between the two classes. Once trained, the model is able to separate text lines as a function of the script. Similar to detection, rather than training the networks from scratch, we fine-tune known pre-trained models (Inception and ResNet) to solve the two-class classification problem.
\begin{figure*}[!ht]
	\centering
		\includegraphics[width=0.95\textwidth]{figure9.pdf}
	\caption{Script identification of detected text lines}
\label{fig:scriptIden}
\end{figure*}

\subsection{Hybrid Text Detector \& Script Identifier}
Detection of text and identification of script, as discussed previously, can be implemented in a cascaded framework where the output of text detector is fed to the script identifier. A deep learning framework can be tuned to discriminate between text and non-text regions and the extracted text regions can be fed to a separate script recognition model that identifies the script of the detected text. This, however, introduces a bottleneck of training two separate networks. Furthermore, the cascaded solution also implies that errors in detection are propagated to the next step as well. We, therefore, propose to combine the text detector and script identifier in a single hybrid model. Rather than treating detection as a two-class problem (text and non-text), we consider it as a three class problem, i.e. non-text regions, English text and Urdu text. This not only avoids training two separate models but also eliminates the accumulation of errors in a cascaded solution. The superiority of the combined text detector and script identifier is also supported through quantitative evaluations as discussed in the next section.\\

All detectors are trained in an end-to-end manner with a multi-task objective function that combines the classification and regression losses. The evolution of training loss for the investigated detectors (with Inception and ResNet) is illustrated in Figure~\ref{fig:alllosses} where it can be seen that the loss begins to stabilize from 30 epochs on wards.

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.75\textwidth]{figure10.pdf}
	\caption{Training loss of various detectors -- Hybrid text detector and script identifier}
\label{fig:alllosses}
\end{figure}

\section{Experiments and Results}\label{sec:Experiments}
The detection performance is evaluated through a series of experiments carried out on the collected set of video frames. We first present the experimental protocol followed by the detection results of various object detectors. We then present the script identification results and the performance of the combined text detector and script identifier. Furthermore, performance sensitivity of the system as well as a comparison with state-of-the-art is also presented.

\subsection{Experimental Settings}
As introduced in Section~\ref{sec:Dataset}, we collected a total of 11,203 video frames from four different News channel videos. The localization information of text regions in these frames is used to train and subsequently evaluate the text detection and script identification performance. The distribution of frames into training and test sets along with the number of text lines in each set is summarized in Table~\ref{tab:dataDist} while the details of detection performance are presented in the next section.

\begin{table}[!ht]
\centering
\caption{Data distribution for text detection experiments}
\label{tab:dataDist}

\begin{tabular}{lllll}
\hline
                                       & \multicolumn{2}{c}{\textbf{Train}}                               & \multicolumn{2}{c}{\textbf{Test}}                                        \\ \hlin
                                       & \multicolumn{1}{c}{\textbf{Frames}} & \multicolumn{1}{c}{\textbf{Lines}} & \multicolumn{1}{c}{\textbf{Frames}} & \multicolumn{1}{c}{\textbf{Line}} \\ \hline
\multicolumn{1}{l}{\textbf{Urdu}}    & \multirow{2}{*}{8,500}               & 31,321                              & \multirow{2}{*}{2,703}               & 9,149                              \\
\multicolumn{1}{l}{\textbf{English}} &                                      & 16,207                              &                                      & 7,425                              \\ \hline
\multicolumn{1}{l}{\textbf{Total}}   &                                      & 49,046                     &                                      & 11,056                    \\ \hline

\end{tabular}
\end{table}

\subsection{Text Detection Results}
Object detectors including Faster R-CNN, YOLO, SSD and R-FCN are adapted to detect textual content by fine-tuning the Inception and ResNet pre-trained models and changing the anchor boxes as discussed previously. Performance of each of these detectors in terms of precision, recall and F-measure is summarized in Table~\ref{tab:detectResText}. It can be seen that in all cases, detectors pre-trained on Inception outperform those trained on ResNet. Among various detectors, Faster R-CNN reports the highest F-measure of 0.90. The lowest performance is reported by Yolo reading an F-measure of 0.66. A comprehensive study on the trade-off between speed and accuracy of various object detectors is presented in~\cite{huang2017speed} and our findings on detection of text are consistent with those of~\cite{huang2017speed}. It is also important to recap that precision and recall are computed using area based metrics. As a result, if the detected bounding box is larger (smaller) than the ground truth, it results in penalizing the precision (recall) of the detector as illustrated in Figure~\ref{fig:prerecall}. The output of the Faster R-CNN based text detector for few sample frames in our dataset is illustrated in Figure~\ref{fig:detectionResultsSamp}.
\\

\begin{table*}[!ht]
\centering
\caption{Text Detection Results}
\label{tab:detectResText}
\begin{tabular}{lcccccc}
\hlin
                                            & \multicolumn{3}{c}{\textbf{RestNet}}                     & \multicolumn{3}{c}{\textbf{Inception}}                   \\ \hline
\multicolumn{1}{l}{\textbf{Model}}        & \textbf{Precision} & \textbf{Recall} & \textbf{F-Measure} & \textbf{Precision} & \textbf{Recall} & \textbf{F-Measure} \\ \hline
\multicolumn{1}{l}{\textbf{SSD}}          & 0.83               & 0.71            & 0.77               & 0.82               & 0.77            & 0.80               \\ \hline
\multicolumn{1}{l}{\textbf{R-FCN}}        & 0.79               & 0.86            & 0.82               & 0.84               & 0.89            & 0.86               \\ \hline
\multicolumn{1}{l}{\textbf{Faster R-CNN}} & 0.82               & 0.90            & 0.85               & \textbf{0.86}      & \textbf{0.95}   & \textbf{0.90}      \\ \hline
\multicolumn{1}{l}{\textbf{Yolo}}         & -                  & -               & -                  & 0.63               & 0.69            & 0.66               \\ \hline
\end{tabular}
\end{table*}

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.75\textwidth]{figure11.pdf}
	\caption{Computation of precision and recall (a):Ground Truth Bounding Box (b): Detected region is larger than ground truth (c):Detected region is smaller than ground truth (d):Detected region overlaps perfectly with the ground truth}
\label{fig:prerecall}
\end{figure}

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.85\textwidth]{figure12.pdf}
	\caption{Text detection results on sample images (Faster R-CNN with Inception) }
\label{fig:detectionResultsSamp}
\end{figure}

In an attempt to provide an insight into the detection errors, few of the errors are illustrated in Figure~\ref{fig:mistakes1}. It can be seen that in most cases, the detector is able to detect the textual region but the localization is not perfect i.e. in some cases the bounding box is larger (shorter) than the actual content leading to a reduced precision (recall). \\

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.85\textwidth]{figure13.pdf}
	\caption{Imperfect Localization of Text Regions}
\label{fig:mistakes1}
\end{figure}

\subsection{Script Identification Results}
For script identification, we employ the same distribution of frames into training and test sets as that of the detection protocol. Text lines from the video frames in the training set are employed to fine-tune the pre-trained ConvNets while the identification rates are computed on text lines from the frames in the test set. A total of $31,321$ Urdu and $16,207$ English text lines are used in the training set while the test set comprises $9,9149$ and $7,425$ text lines in Urdu and English respectively. The resulting confusion matrix is presented in Table~\ref{tab:siConMat} while the precision, recall and F-measure are summarized in Table~\ref{tab:sifmeasure}. It can be seen that the model was able to correctly identify the scripts with an accuracy of more than 94\%.\\

\begin{table}[!ht]

\centering

\caption{Script identification confusion matrix}
\label{tab:siConMat}
\begin{tabular}{lcc}
\hline
& \textbf{Urdu} & \textbf{English} \\
\hlin
\textbf{Urdu}  & 8763 & 386 \\
\textbf{English} & 551 & 6874 \\
\hline
\end{tabular}
\end{table}

\begin{table}[!ht]
\centering
\caption{Performance of Script Identification }
\label{tab:sifmeasure}
\begin{tabular}{lccc}
\hline

 &	\textbf{Precision} &	\textbf{Recall} & 	\textbf{F--Measure} \\ \hline

\textbf{Urdu} 	& 0.940 &	0.957 & 0.95\\
\textbf{English} & 0.946 & 0.925 & 	0.94 \\
\hline
\end{tabular}
\end{table}

\subsection{Hybrid Text Detection \& Script Identification Results}
As discussed previously, text detection and script identification can be combined in a single model treating detection as a three (rather than two) class problem. The results of these experiments are summarized in Table~\ref{tab:detectRes} keeping the same distribution of training and test frames as in the previous experiments. Many interesting observations can be drawn from the results in Table~\ref{tab:detectRes}. Similar to the script independent detectors, models pre-trained on Inception outperform those trained on ResNet and the observation is consistent for all four detectors. Likewise, Faster R-CNN reports the highest F-measure both for detection of Urdu and English text reading 0.91 and 0.87 respectively. In all cases, the performances on detection of Urdu text are better than those on Engish text. This can be attributed to the fact that the data is collected primarily from Urdu News channels which have limited amount of English text. It is also interesting to note that by combining text detection and script identification in a single model, not only the cascaded solution is avoided, the detection F-measures have also improved (in most cases). Though the improvement is marginal, eliminating the separate processing of detected text regions to identify the script offers a much simplified (yet effective) solution. Detection outputs on sample frames for the four detectors are illustrated in Figure~\ref{fig:alldetectors}.

\begin{table*}[!ht]
\centering
\caption{Performance of hybrid text detector and script identifier}
\label{tab:detectRes}
\begin{tabular}{llcccccc}
\hlin
                                                             &                  & \multicolumn{3}{c}{\textbf{RestNet}}                     & \multicolumn{3}{c}{\textbf{Inception}}                   \\ \hline
\multicolumn{1}{l}{\textbf{Method}}                         & \textbf{Script}  & \textbf{Precision} & \textbf{Recall} & \textbf{F-Measure} & \textbf{Precision} & \textbf{Recall} & \textbf{F-Measure} \\ \hline
\multicolumn{1}{l}{\multirow{3}{*}{\textbf{SSD}}}          & \textbf{Urdu}    & 0.83               & 0.72            & 0.77               & 0.82               & 0.78            & 0.80               \\
\multicolumn{1}{l}{}                                       & \textbf{English} & 0.80               & 0.63            & 0.70               & 0.82               & 0.70            & 0.75               \\
\hline
\multicolumn{1}{l}{\multirow{3}{*}{\textbf{R-FCN}}}        & \textbf{Urdu}    & 0.80               & 0.87            & 0.83               & 0.85               & 0.90            & 0.87               \\
\multicolumn{1}{l}{}                                       & \textbf{English} & 0.73               & 0.81            & 0.77               & 0.77               & 0.84            & 0.81               \\
\hline
\multicolumn{1}{l}{\multirow{3}{*}{\textbf{Faster R-CNN}}} & \textbf{Urdu}    & 0.82               & 0.92            & 0.86               & \textbf{0.87}      & \textbf{0.95}   & \textbf{0.91}      \\
\multicolumn{1}{l}{}                                       & \textbf{English} & 0.80               & 0.81            & 0.80               & \textbf{0.81}      & \textbf{0.94}   & \textbf{0.87}      \\
\hline
\multicolumn{1}{l}{\multirow{3}{*}{\textbf{Yolo}}}         & \textbf{Urdu}    & -                    &   -              &    -                & 0.64               & 0.70            & 0.67               \\
\multicolumn{1}{l}{}                                       & \textbf{English} &  -                  &    -             &    -               & 0.62               & 0.67            & 0.64               \\
\hline
\end{tabular}
\end{table*}

\begin{figure*}[!ht]
	\centering
		\includegraphics[width=0.85\textwidth]{figure14.pdf}
	\caption{Detection output of hybrid text detection and script identification for different detectors (a): SSD (b): R-FCN (c): Faster RCNN (d): Yolo }
\label{fig:alldetectors}
\end{figure*}

In an attempt to carry out an in-depth analysis of the detection performance and its evolution with respect to important system parameters, we carried out another series of experiments using Faster R-CNN (with Inception). In the first such experiment, we study the performance sensitivity to the amount of training data. We train the model by varying the number of text line images (from 10K to 49K) and compute the detector F-measure. Naturally, the detector performance enhances with the increase in the amount of training data (Figure~\ref{fig:impact}) and begins to stabilize from around 30K-35K training lines.

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.75\textwidth]{figure15.pdf}
	\caption{Impact of size of training data on text detection performance (Faster R-CNN with Inception)}
\label{fig:impact}
\end{figure}

Resolution of input video frames is an important parameter that might affect the detector performance. To study the detector sensitivity to image resolution, we varied the image resolution from $256 \times 144$ to $1920 \times 1080$. The resolution was varied only in the test set and all sets of images were evaluated on the detector trained on a single resolution ($900 \times 600$). The F-measures in Figure~\ref{fig:resolution} are more less consistent for varied image resolutions reflecting the robustness of the detector. The proposed anchor boxes adapted for textual content play a key role in achieving this scale invariance.

\begin{figure}[!ht]
	\centering
		\includegraphics[width=0.75\textwidth]{figure16.pdf}
	\caption{Impact of video resolution on text detection performance (Faster R-CNN with Inception)}
\label{fig:resolution}
\end{figure}

\subsection{Performance Comparison}
In an attempt to compare the performance of our detector with those reported in the literature, we present a comparative overview of various text detectors targeting cursive caption text in Table~\ref{tab:detectResCom}. It is important to note that since different studies are evaluated on different datasets, a direct comparison of these techniques is difficult. Most of the listed studies employ a small set of images ($\leq 1000$). Moradi et al.~\cite{moradi2013hybrid} and Zayene et al.~\cite{Oussama2016} report results on relatively larger datasets with F-measures of 0.89 and 0.84 respectively. In comparison to other studies, we employ a significantly larger set of images with an F-measure of 0.91. Furthermore, for a fair comparison, we also evaluated our system on the set 1000 images in the publicly available IPC dataset~\cite{siddiqi2012database}, the corresponding F-measure reads 0.92 validating the effectiveness of our detection technique,

\begin{table*}[!ht]
\centering
\caption{Performance comparison with other techniques}
\label{tab:detectResCom}
\footnotesize
\begin{tabular}{llllcccc}
\hline
\multicolumn{1}{l}{\textbf{Study}}                              &
\multicolumn{1}{l}{\textbf{Method}} &
\multicolumn{1}{l}{\textbf{Dataset}}                              &
\multicolumn{1}{l}{\textbf{Script}}   & \multicolumn{1}{c}{\textbf{Video Frames}} & \multicolumn{1}{c}{\textbf{Precision}} & \multicolumn{1}{c}{\textbf{Recall}} & \multicolumn{1}{c}{\textbf{F-Measure}} \\ \hline

Jamil et al.(2011)~\cite{jamil2011edge}   & Edge-based Features   & IPC & Urdu    & 150    & 0.77  & 0.81      & 0.79          \\
Siddiqi and Raza(2012)~\cite{siddiqi2012database} & Image Analysis & IPC & Urdu    & 1,000   & 0.71  & 0.80     & 0.75            \\
Moradi et al.(2013)~\cite{moradi2013hybrid}   & LBP with SVM & -    & Farsi/Arabic & 4971 & 0.91 & 0.87   & 0.89          \\
Raza et al.(2013)~\cite{raza2013multilingual} & Cascade of Transforms  &  IPC  & Urdu    & 1,000   & 0.80      & 0.89   & 0.84       \\
Raza et al.(2013)~\cite{raza2013multilingual}& Cascade of Transforms  &  IPC    & Arabic & 300 & 0.81  & 0.93    & 0.86      \\
Yousfi et al.(2014)~\cite{yousfi2014arabic}   & ConvNet    & - & Arabic & 201     & 0.75    & 0.80     & 0.77                  \\
Zayene et al.(2015)~\cite{Oussama2015}    & SWT & AcTiV        & Arabic  & 425     & 0.67   & 0.73     & 0.70                   \\
Zayene et al.(2016)~\cite{Oussama2016}    & SWT\&Conv Autoencoders  & AcTiV  & Arabic   & 1843    & 0.83      & 0.85     & 0.84        \\
Shahzad et al.(2017)~\cite{shahzad2017oriental}  & Image Analysis & - & Urdu/Arabic & 240 & 0.83   & 0.93   & 0.88                  \\
Mirza et al.(2018)~\cite{mirza2018urdu}  & Textural Features & UTiV & Urdu  & 1,000    & 0.72   & 0.89    & 0.80           \\
Unar et al.(2018)~\cite{unar2018artificial}  & Image Analysis+SVM & IPC     & Urdu   & 1,000  & 0.83    & 0.88     & 0.85            \\
\textbf{Proposed Method}    & \textbf{Deep ConvNets} & UTiV  & \textbf{Urdu}&    \textbf{11,203} & \textbf{0.87}  & \textbf{0.95} & \textbf{0.91}\\
& & IPC  & \textbf{Urdu} &    \textbf{1,000} & \textbf{0.91}  & \textbf{0.93} & \textbf{0.92}\\

\hline

\end{tabular}
\end{table*}

\section{Conclusion}\label{sec:Conclusion}
This paper presented a system for detection of caption text appearing in video frames. The developed technique relies on exploiting deep learning based object detectors and adapting them for text detection. Since it is common in videos to have text in more than one script, we presented, as a case study, video frames with text in cursive (Urdu) and Roman (English) scripts. Since each script requires different processing, the detection is combined with script identification in an end-to-end fashion so that the system is able to not only localize the text but also identify its script. Among various investigated object detectors, Faster R-CNN with our proposed set of anchor boxes reported the highest detection rates.\\

The presented work is a part of a comprehensive video indexing and retrieval system and the current study focused on the detection of text. In our other~\cite{ali1,ali2} work, the detection module is  integrated with the video OCR module so that detected text is recognized. Once recognized, videos are indexed based on keywords and retrieved on user provided queries. In addition to retrieval, automatic News summarization from ticker text and comparison of News across various News channels is also planned to be implemented. Furthermore, the textual content based retrieval will be combined with audio as well as visual objects appearing in videos.


\section{Introduction}

 Let $A\in  B(\ell_2({\Bbb Z}))$ be a bounded operator on $\ell_2({\Bbb Z})$, we can write $A$ in its matrix presentation
   $A=(a_{k,j})$ with $a_{k,j}=\langle Ae_k,e_j\rangle $.
Given a bounded function $m$ on ${\Bbb Z}\times {\Bbb Z}$,
 we call the map
 \begin{eqnarray}
 S_m: (a_{k,j})\mapsto (m_{k,j}a_{k,j})\label{sm}
 \end{eqnarray}
  a Schur multiplier with symbol $m$.
A historic question is to find conditions on the symbol $m$ so that $S_m$ is bounded with respect to the Schatten $p$-norm of $A$. Here the Schatten-$p$ norm  is defined as
\begin{eqnarray*}
\|A\|_p&=&(tr (A^*A)^{\frac p2})^{\frac1p}, \ \ p<\infty\\
 \|A\|_\infty&=&\sup_ {\|x\|_{\ell_2}=1} \|Ax\|_{\ell_2}.
 \end{eqnarray*}
 Using the well-know connection between Fourier multiplier on operator-valued functions and Toeplitz type Schur multipliers, J. Bourgain (\cite{Bou86}) proved a Marcinkiewitz type theory for Toeplitz type Schur multpliers.
 The main result of this article extends J. Bourgain's result to  non-Toeplitz type Schur multipliers.

\begin{theorem}
\vskip.1cm
 $S_m$ defined in (\ref{sm}) extends to a completely bounded map on $S_p$ for all $1<p<\infty$ with bounds $\lesssim (\frac{p^2}{p-1})^3$, if  $m$ is bounded and there exists a constant $C$ such that
 \begin{eqnarray}
  \sum_{2^{N-1}\leq |k|< 2^{N}}|m(k+j+1,j)-m(k+j,j)| <C \label{Marc}\\
 \sum_{2^{N-1}<|k|<2^{N}}|m(j,k+j+1)-m(j,k+j)| <C \label{Marr},
\end{eqnarray}
for all $N\in{\Bbb N}$.
 \end{theorem}

  Hyt\"onen and Weiss (\cite{HW08}) have showed that for operator valued multipliers on a Banach space $X$, the natural Marcinkiewicz Fourier multiplier theory holds if and only if the Banach space $X$ satisfies Pisier's property $\alpha$. While Schatten-$p$ class fails the property $\alpha$,   this article shows that  Marcinkiewicz  type Schur multiplier theory still holds.

The writing of this article was motivated by the recent work by Conde etc. (\cite{CondeAlonso2022}), although the third author had known Theorem 1.1 for a while.  Conde etc. proved in \cite{CondeAlonso2022}   a  remarkable H\"ormander-Mikhlin type criterion for Schur mulipliers on $S_p({\Bbb R}^n)$. The   considering of multi-indexed $S_p$ is motivated by the recent work of  Lafforgue/de la Salle (\cite{Lafforgue2011}) on lattices of higher rank Lie groups (e.g. $SL_3({\Bbb Z}$)), in which  Schur multipliers on multi-indexed Schatten $p$-classes play important roles. Following this trend, we extend our result to higher dimension ${\Bbb Z}^n$ and the case of continuous indices ${\Bbb R}^n$ as well.

\section{Preliminaries}
Denote $\mathcal{B}(\ell^2(\mathbb{Z}^d))$  the set of bounded linear operators on $\ell^2(\mathbb{Z}^d)$. We present operator $A\in \mathcal{B}(\ell^2)$ as $A=(a_{i,j})_{(i,j)\in \mathbb{Z}^d\times \mathbb{Z}^d}$ with $a_{i,j}=\langle Ae_k,e_j\rangle $ for a fixed orthogonal basis $e_i$ of $\ell^2(\mathbb{Z}^d)$

The Schur multiplier $M=(m_{i,j})_{(i,j)\in \mathbb{Z}\times \mathbb{Z}}$ acting on the the $\mathcal{B}(\ell^2(\mathbb{Z}^d))$ is the weak* continuous map on $\mathcal{B}(\ell^2{(\mathbb{Z})}))$ extending the map
\[
	S_M(A)=(m_{ij})\circ(a_{ij})=(m_{ij}a_{ij})
\]
for operators $A=(a_{i,j})$ on the Schatten class $S_2(\ell^2(\mathbb{Z}^d))$.

For $f\in L^2(\mathbb{T}^d)$ and $R$ a subset of ${\Bbb Z}^d$, denote $S_{R} f$ the   partial Fourier sum
\[
	S_{R} f(z)= \sum_{n\in R} \hat{f}(n) z^n.
\]

\begin{comment}
Suppose $\eta$ is a $C^{\infty}$ function of compact support on $\mathbb{R}$ with the properties that $\eta(x)=1$ for $|x|\leq 1$ and $\eta(x)=1$ for $|x|\geq 2$. Let $\delta(x)=\eta(x)-\eta(2x)$, then $\delta$ is also a $C^{\infty}$ function with compact support on $[-2,-\frac{1}{2}]\cup [\frac{1}{2}, 2]$. Then for any $x\in\mathbb{R}$, there is
\[
	\sum_{k=-\infty}^{\infty} \delta(2^{-j}x)=1. \quad \text{all~}x\neq 0.
\]
\end{comment}
Choose $\delta_d\in C^{\infty}(\mathbb{R})$ such that   $0\leq\delta_d\leq 1$, $supp(\delta)\subset [-2\sqrt d ,-\frac{1}{4}]\cup[\frac{1}{4},2\sqrt d]$ and
 $\delta(x)=1$ when $\frac12\leq |x|\leq \sqrt d$. For $j\in\mathbb{Z}$, define $\delta_j(x):=\delta(2^{-j}x)$. For $f\in L^2(\mathbb{T}^d)$, we define
 \[
 S_{\delta_j}f(z)=\sum_{n\in\mathbb{Z}^d} \delta_j(|n|_{2}) \hat{f}(n) z^n.\]
Let $(E_j)_{j\geq 0}$ be the cubs with squared holes in $\mathbb{Z}^d$  given by
\begin{equation}\label{eq:dyadic}
E_j =
\begin{cases}
		\{n\in\mathbb{Z}^d: 2^{j-1}< |n|_\infty\leq 2^j\} & j\geq 0,\\
		\{0\} & j=0,
	\end{cases}
\end{equation} Note our construction implies $ S_{E_j}S_{\delta_j}=S_{E_j}$ which we will use frequently.
 We will need   the following noncommutative Littlewood-Paley theorem on $\mathbb{Z}^d$.
\begin{lemma}\label{thm:Littwood-P}
Suppose $f(z)=\sum_{n\in \mathbb{Z}^d}\hat{f}(n)z^n\in L^{\infty}(\mathbb{T}^d)\otimes \mathcal{B}(\ell^2(\mathbb{Z}^d))$ where $\hat{f}(n)\in \mathcal{B}(\ell^2(\mathbb{Z}^d))$ and $p\in (1,\infty)$. Denote $\mathcal{M}= L^{\infty}(\mathbb{T}^d)\bar{\otimes} \mathcal{B}(\ell^2(\mathbb{Z}^d))$. There is
\begin{align}
\|f\|_{L^p(\mathbb{T}^d;S_p)} &\leq C \frac{p^2}{p-1} \left\| (S_{E_j} f )\right\|_{L^p(\mathcal{M},\ell^2_{cr})}, \\
\left\| (S_{\delta_j} f )\right\|_{L^p(\mathcal{M},\ell^2_{cr})} &\leq C \frac{p^2}{p-1}\|f\|_{L^p(\mathbb{T}^d;S_p)}.
\end{align}
Here $C$ is a constant independent of $p$ and $d$.
\end{lemma}

\begin{comment}
\begin{theorem}[Marcinkiewicz,\cite{Marcinkiewicz1939}]
Let $(\Delta_j)_{j\in\mathbb{Z}}$ be the dyadic decomposition of  $\mathbb{Z}$. Suppose $M(n)$ is a function on $\mathbb{Z}$ such that
\[
	\sup_{n\in\mathbb{Z}} |M(n)|\leq C,\quad \sup_{j\in\mathbb{Z}}\sum_{n\in\Delta_j}|\Delta M(n)|\leq C
\]
where $\Delta M(n)=M(n+1)-M(n)$. Then
\[
	\|T_M f\|_p \leq C_p C \|f\|_p
\]
for every $p\in (1,\infty)$, where $C_p$ is a number depending only on $p$.
\end{theorem}
\end{comement}

\begin{comment}
{
	\begin{lemma}\label{thm:Littwood-P}
		Suppose $f(z)=\sum_{n\in \mathbb{Z}}\hat{f}(n)z^n\in L^{\infty}(\mathbb{T})\otimes \mathcal{B}(\ell^2(\mathbb{Z}))$ where $\hat{f}(n)\in \mathcal{B}(\ell^2(\mathbb{Z}))$ and $p\in (1,\infty)$, then
		\[
		\|f\|_{L^p(\mathbb{T};S_p)} \asymp
		\begin{cases}
		\inf\left\{ \left\|\left(\sum_{j\in\mathbb{Z}}g_j g_j^*\right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}+\left\|\left(\sum_{j\in\mathbb{Z}}h_j^* h_j\right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}:S_{j}f=g_j+h_j\right\} & p\in(1,2]\\
   \left\| \left(\sum_{j\in\mathbb{Z}} (S_jf)^*(S_jf)\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)}+ \left\| \left(\sum_{j\in\mathbb{Z}} (S_jf) (S_jf)^*\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} & p\in [2,\infty).
		\end{cases}
   \]
	\end{lemma}
}
\end{comment}

\begin{lemma}\label{Hilbert}
Suppose $R_j$ is a family a rectangles with sides parallel to the axes in $\mathbb{R}^n$. Then there is a constant $C_d>0$ such that for all $1<p<\infty$, for all $\sigma$-finite measures $\mu$ on $\mathbb{R}^d$, and for all families of measurable functions $f_j$ on $\mathbb{R}^d$, we have
\begin{equation}
	\left\| \sum_j | (\chi_{R_j} \widehat{f}_j)^{\vee}|^2 )^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} \leq C_d \left(\frac{p^2}{p-1}\right)^d \left\| \left( \sum_j |f_j|^2\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)}.
\end{equation}
\end{lemma}

Also we are going to use the following Cauchy-Schwarz type lemma.

	\begin{lemma}\label{lem:Cauchy-S}
		Suppose $a_n,c_n\in \mathcal{B}(\ell^2(\mathbb{Z}^d)), n\in\mathbb{Z}^d$. Then
\begin{equation}\label{eq:C-S}
	\left|\sum_{n\in\mathbb{Z}^d} a_n^* c_n\right|^2 \leq \left\|\sum_{n\in\mathbb{Z}^d} |a_n|^2 \right\|_{\infty} \left( \sum_{n\in\mathbb{Z}^d} |c_n|^2\right)
\end{equation}
	\end{lemma}
	\begin{proof}
		It is equivalent to prove that for any $v\in \ell^2(\mathbb{Z}^d)$, there is
		\begin{equation}\label{eq:inner}
		\langle |\sum_n a_n^* c_n|^2 v, v\rangle \leq \|\sum_n |a_n|^2\|_{\infty}\langle \sum_{n}|c_n|^2 v,v  \rangle.
		\end{equation}
		Suppose $(e_{k,l})_{k,l\in\mathbb{Z}^d}$ is the canonic basis of $\mathcal{B}(\ell^2(\mathbb{Z}^d))$ and denote $A=\sum_{n} a_n^*\otimes e_{n,1}$, $C=\sum_n c_n v \otimes e_{1,n}$. Then
		\[
		\text{LHS of \eqref{eq:inner}}= \|\sum_{n} a_n^* c_n v \|^2= \|AB\|^2 \leq \|AA^*\| \|C^*C\|\leq \text{RHS of \eqref{eq:inner}}.
		\]
		Hence, the lemma is proved.
	\end{proof}
}

 \section{The Schur-Marcinkiewicz multiplier theorem on ${\mathbb{Z}}$}

\begin{theorem}\label{thm:main}
 Given $M=(m_{ij})_{i,j\in \mathbb{Z}}\in \mathcal{B}(\ell^2(\mathbb{Z}))$.
  Suppose $M$ satisfies that
\begin{enumerate}
 \item[i)]  $\sup_{i,j\in \mathbb{Z}}|m_{ij}|<C_1$ and
\item[ii)]  For any $k\in\mathbb{N}, s\in\mathbb{Z}$, there are constants $C_1,C_2$ such that
\begin{equation}\label{eq:rowAndColumn}
 \sum_{2^{k-1}\leq |j| <2^k}|m_{i+j+1,i}-m_{i+j,i}|<C_2, \quad \sum_{2^{k-1}\leq |j| <2^k} |m_{i,i+j+1}-m_{i,i+j}|<C_2,
\end{equation}
\end{enumerate}

Then $M$ is a (completely) bounded Schur multiplier on $S_p(\ell^2(\mathbb{Z}))$ for $p\in(1,\infty)$  with the bound $C_3 (\frac{p^2}{p-1})^3$, where $C_3$ is an absolute constant.
\end{theorem}
\begin{proof}Define the *-homomorphism $\pi: \mathcal{B}(\ell^2(\mathbb{Z}^d))\to L^{\infty}(\mathbb{T}^d)\otimes \mathcal{B}(\ell^2(\mathbb{Z}^d))$ by
\begin{equation}\label{eq:transference}
	\pi: A=(a_{st})\longmapsto (a_{st} z^{s-t}).
\end{equation}

Given $M=(m_{ij})_{i,j\in \mathbb{Z}^d}, A=(a_{ij})_{i,j\in\mathbb{Z}}\in \mathcal{B}(\ell^2(\mathbb{Z}^d ))$, denote
\begin{align}\label{eq:muliplierNotation}
&M_{l}(n)=\sum_{s\in\mathbb{Z}^d}m_{s,s-n}\, e_{s,s},\quad M_r(n)=\sum_{s\in \mathbb{Z}^d } m_{s+n,s}\,e_{s,s},\\
&A_l(n)=\sum_{s\in\mathbb{Z}^d } a_{s,s-n} e_{s,s-n}, \quad A_r(n)=\sum_{s\in\mathbb{Z}^d } a_{s+n,s} e_{s+n,s}. \nonumber
\end{align}

By the notations in \eqref{eq:muliplierNotation}, the condition \eqref{eq:rowAndColumn} is equivalent to
\begin{equation}
	 \sup_{j\in \mathbb{Z}} \left\|\sum_{n\in E_j} |M_l(n+1)-M_l(n)| \right\|_{\infty}<C, \quad \sup_{j\in \mathbb{Z}} \left\|\sum_{n\in E_j} |M_r(n+1)-M_r(n)| \right\|_{\infty}<C,
\end{equation}

Let
\begin{equation}\label{f}
f(z)=\pi(A)
=\sum_{n\in\mathbb{Z}} A_l(n)z^n =\sum_{n\in\mathbb{Z}} A_r(n)z^n.
\end{equation}
Let
\begin{equation}\label{eq:transfer}
	T_{\widetilde{M}} \, \pi (A)=\pi S_M (A) .
\end{equation}
It can be easily verified that
\begin{align}\label{eq:transf1}
	T_{\widetilde{M}} \, \pi (A)&= \sum_{n\in\mathbb{Z}^d} M_l(n)A_l(n)z^n\\
	&= \sum_{n\in\mathbb{Z}^d } A_r(n) M_r(n) z^n, \label{eq:transf2}
\end{align}
Since $\pi$ is a $*$-homomorphism then
\begin{equation}
	\|A\|_{S_p}=\|f(z)\|_{L^p(\mathbb{T};S_p)}, \ \ \|S_MA\|_{S_p}=\|T_{\widetilde{M}}(f)\|_{L^p(\mathbb{T};S_p)}.
\end{equation}
In order to prove $M$ is a bounded Schur multiplier, we need to prove that $T_{\widetilde{M}}$ is  bounded  on the subspace of $L_p(\mathbb{T};S_p)$ consisting all $f$ in the form of (\ref{f}). By noncommutative Littlewood-Paley theorem \eqref{thm:Littwood-P} and the transference relation \eqref{eq:transf1},\eqref{eq:transf2}, it is sufficient to show that
\begin{equation}\label{eq:L-P1}
	\left\| \left( \sum_{j\in\mathbb{Z}} |S_{E_j}(T_{\widetilde{M}}f)|^2 \right)^{\frac12} \right\|_{L_p(\mathbb{T};S_p)}\leq C \frac{p^2}{p-1} \left\| \left( \sum_{j\in\mathbb{Z}} |S_{\delta_j}f|^2 \right)^{\frac12} \right\|_{L_p(\mathbb{T};S_p)}
\end{equation}
and its adjoint form for $p\geq 2$, where $C$ is a positive constant.

Denote the frequency projection operator $S_{(a,b)}$ for the $g(z)=\sum_{n\in\mathbb{Z}} \hat{g}(n)z^n\in L^{\infty}(T)\otimes~\mathcal{B}(\ell^2(\mathbb{Z}))$ as follows.
\begin{equation}
	S_{(a,b)}g(z)= \sum_{a<n<b} \hat{g}(n)z^n
\end{equation}
Write $E_j=E_{j,1}\cup E_{j_2}$ with $E_{j,1}=E_j\cap {\Bbb Z}_-, E_{j,2}=E_j\cap {\Bbb Z}_+$. Let $a_{j,1}=-2^{j},a_{j,2}=2^{j}$ and
\begin{eqnarray}
\Delta M(n)=sgn(n)(M(n+1)-M(n))
\end{eqnarray}

Then via summation by parts and notice that $S_{(2^{j-1}-1,2^{j}-1)}f=S_{E_j}f$ and $S_{(2^{j+1}-1,2^{j+1})}f=0$,
\begin{align}\label{eq:summationByPart1}
	S_{E_j}(T_{\widetilde{M}}f) =\sum_{i=1,2}S_{E_{j,i}}(T_{\widetilde{M}}f) =\sum_{i=1,2} \left(M_l(a_{j,i}) (S_{E_{j,i}} f)+\sum_{n\in E_{j,i}} \Delta M_l(n) (S_{(n,a_{j,i})}f)\right)
\end{align}
We will use the presentation (\ref{eq:summationByPart1}) to prove (\ref{eq:L-P1}), and use (\ref{eq:summationByPart2})
\begin{align}
	S_{E_j}(T_{\widetilde{M}}f) =\sum_{i=1,2}S_{E_{j,i}}(T_{\widetilde{M}}f) =\sum_{i=1,2} \left((S_{E_{j,i}} f)M_r(a_{j,i}) +\sum_{n\in E_{j,i}}  (S_{(n,a_{j,i})}f)\Delta M_r(n)\right) \label{eq:summationByPart2}
\end{align}
	to prove the adjoint version of (\ref{eq:L-P1}). This will help to reduce the usual difficulties caused by the noncommutativity.

Note  $\Delta M_l(n)$ is a diagonal operator, we can write  $\Delta M_l (n)=a^*_n b_n $ with $a_n,b_n$ diagonal operators and $|a_n|^2=|b_n|^2=|\Delta M_l(n)| $,
then by Lemma \ref{lem:Cauchy-S}, we have
\begin{align}
	\left|\sum_{n\in E_j}\Delta M(n) S_{(n,a_{j,i})} f\right|^2 &\leq \left\| \sum_{n\in E_j} |\Delta M_l(n)| \right\|_{\infty} \left( \sum_{n\in E_j}|b_nS_{(n,a_{j,i})}f|^2 \right) \nonumber \\

	&\leq C_2   \left( \sum_{n\in E_j}|S_{(n,a_{j,i})}(b_nS_{\delta_j}f)|^2 \right).\label{eq:sumP2}
\end{align}

\noindent Thus, by Lemma \ref{Hilbert}
\begin{align}
 \left\| \left( \sum_{j\in\mathbb{Z}}\left|\sum_{n\in E_{j,i}}\Delta M(n) S_{(n,a_{j,i})} f\right|^2 \right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}

 &\leq C_2^{\frac{1}{2}} \left\| \left( \sum_{j\in\mathbb{Z}}\sum_{n\in E_{j,i}}\left|  S_{(n,a_{j,i})} \left(b_n S_{\delta_j} f\right)\right|^2\right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}
 \nonumber \\
 &\leq C_3\frac{p^2}{p-1} C_2^{\frac{1}{2}}  \left\| \left( \sum_{j\in\mathbb{Z}}\sum_{n\in E_{j,i}}\left|  \left(b_n S_{\delta_j} f\right)\right|^2\right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}
 \nonumber \\
 &=  C_3\frac{p^2}{p-1} C_2^{\frac{1}{2}}  \left\| \left( \sum_{j\in\mathbb{Z}} (S_{\delta_j}f)^*  \left(\sum_{n\in E_{j,i}}  |b_n|^2\right) S_{\delta_j}f \right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}   \nonumber\\
 &=  C_3\frac{p^2}{p-1} C_2^{\frac{1}{2}}  \left\| \left( \sum_{j\in\mathbb{Z}} (S_{\delta_j}f)^*  \left(\sum_{n\in E_{j,i}}  |\Delta M_l(n)|\right) S_{\delta_j}f \right)^{\frac12} \right\|_{L^p(\mathbb{T};S_p)}   \nonumber\\
 & \leq C_3\frac{p^2}{p-1}C_2 \left\|\left( \sum_{j\in\mathbb{Z}} | S_{\delta_j} f|^2\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} \label{eq:sumP3}
\end{align}

Hence, by \eqref{eq:summationByPart1}, \eqref{eq:sumP3}, and Lemma \ref{Hilbert}
\begin{align}\label{eq:normSum}
	\left\|\left(\sum_{j\in\mathbb{Z}} |S_{E_{j,i}} (T_{\widetilde{M}} f)|^2 \right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} &\leq
	\left\|  \left(\sum_{j\in\mathbb{Z}} \left|  M_l(a_{j,i})(S_{E_{j,i}} f)\right|^2\right)^\frac12\right\|_{L^p(\mathbb{T};S_p)} +C_3\frac{p^2}{p-1}C_2 \left\|\left( \sum_{j\in\mathbb{Z}} | S_{\delta_j} f|^2\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} \nonumber\\
	 &\leq\left\|C_1 \left(\sum_{j\in\mathbb{Z}} \left| (S_{E_{j,i}} f)\right|^2\right)^\frac12\right\|_{L^p(\mathbb{T};S_p)} +C_3\frac{p^2}{p-1}C_2 \left\|\left( \sum_{j\in\mathbb{Z}} | S_{\delta_j} f|^2\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} \nonumber\\
	 &=\left\|C_1 \left(\sum_{j\in\mathbb{Z}} \left| (S_{E_{j,i}}S_{\delta_j} f)\right|^2\right)^\frac12\right\|_{L^p(\mathbb{T};S_p)} +C_3\frac{p^2}{p-1}C_2 \left\|\left( \sum_{j\in\mathbb{Z}} | S_{\delta_j} f|^2\right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)} \nonumber\\
	&\leq C \frac{p^2}{p-1} \left\|\left(\sum_{j\in\mathbb{Z}} |S_{\delta_j}  f)|^2 \right)^{\frac12}\right\|_{L^p(\mathbb{T};S_p)}
\end{align}
for $i=1,2$.
Therefore we finish the proof of  (\ref{eq:L-P1}). Its adjoint form can be proved similarly. We then complete the proof Theorem \ref{thm:main}.

\end{proof}

\begin{comment}

by the approximation of de la Salle (need to cite!!!),ect, we have ...

\begin{theorem}
	Continuous case for one dimension, to be added...
\end{theorem}

\begin{remark}	to be added...
	1. some examples (Bourgain's result)
	2. connection with
\end{remark}
\end{comment}

\section{The Schur-Marcinkiewicz multiplier theorem on ${\mathbb{Z}}^d$}
In this part, we will generalize Theorem \ref{thm:main} to the $d$-dimensional case. Before we come to the main statement of the theorem, we need to borrow some notations from from calculus of finite differences.

\begin{definition} Let $\sigma:\mathbb{Z}^d\to \mathbb{C}$ and ${j}=(j_1,\cdots,j_n)\in\mathbb{Z}^d$. Let $\{e_j\}_{j=1}^d$ be standard basis of $\mathbb{Z}^d$, i.e., the $j$-th entry of $e_j$ is 1 and 0 for other entries, for $j=1,\cdots,d$. We define the forward and backward partial difference operators $\Delta_{\xi_j}$ and $\widebar{\Delta}_{\xi_j}$, respectively, by
\begin{equation}\label{eq:differenceNotion}i
	\Delta_{\xi_j} \sigma (\xi) \coloneqq \sigma(\xi+ e_j)-\sigma(\xi), \quad
	\widebar{\Delta}_{\xi_j} \sigma (\xi)  \coloneqq \sigma (\xi)- \sigma(\xi-e_j).
\end{equation}
and for $\mathbf{\alpha}\in \mathbb{N}_0^n$, define
\begin{equation*}
	\Delta_{\mathbf{\xi}}^{\mathbf{\alpha}} \coloneqq \Delta_{\xi_1}^{\alpha_1}\cdots \Delta_{\xi_n}^{\alpha_n},\quad
	\widebar{\Delta}_{\mathbf{\xi}}^{\mathbf{\alpha}} \coloneqq \widebar{\Delta}_{\xi_1}^{\alpha_1}\cdots \widebar{\Delta}_{\xi_n}^{\alpha_n}.
\end{equation*}
\end{definition}
Readers can finde more information of calculus of finite differences in chapter 3 of \cite{Ruzhansky2010}. For convenience, we list the following properties of finite differences.
\begin{itemize}
	\item $\Delta_{\xi}^{\alpha} \Delta_{\xi}^{\beta}= \Delta_{\xi}^{\beta}\Delta_{\xi}^{\alpha}=\Delta_{\xi}^{\alpha+\beta}$,
	\item Let $\phi:\mathbb{Z}^D\to\mathbb{C}$, then
	$
	\Delta_{\xi}^{\alpha}(\xi)= \sum_{\beta\leq \alpha} (-1)^{|\alpha-\beta|} {\alpha \choose \beta} \phi(\xi+\beta)
	$
	\item (Discrete Leibniz formula) Let $\phi, \psi:\mathbb{Z}^d\to \mathbb{C} $. Then \[\Delta_{\xi}^{\alpha}(\phi\psi)(\xi)= \sum_{\beta\leq \alpha} {\alpha \choose \beta} (\Delta_{\xi}^{\beta}\phi(\xi)) \Delta_{\xi}^{\alpha-\beta} \psi(\xi+ \beta).\]
\end{itemize}

\subsection{two dimensional case}
Denote a partition $\mathscr{I}^2 \coloneqq \{ E_j: j\geq 0\}	$ of $\mathbb{Z}^2$ as in the case of $\mathbb{Z}$, where $E_j$ is defined as follows
\begin{equation}\label{eq:partitionZ2}
	E_j=\begin{cases}
		 \{(0,0) \}  & j=0,\\
		 \{(n_1,n_2)\in\mathbb{Z}^2:2^{j-1}\leq |(n_1,n_2)|_{\infty}<2^j \} & j\geq 1.
	\end{cases}
\end{equation}
\begin{comment}
Thus, we would have the following noncommutative Littlewood-Paley theorem on $\mathbb{Z}^2$.
\begin{lemma}\label{thm:Littwood-P2}
		Suppose $f(z)=\sum_{n\in \mathbb{Z}^2}\hat{f}(n)z^n\in L^{\infty}(\mathbb{T}^2)\otimes \mathcal{B}(\ell^2(\mathbb{Z}))$ where $\hat{f}(n)\in \mathcal{B}(\ell^2(\mathbb{Z}^2))$ and $p\in (1,\infty)$. Denote $S_{E_j}$ the partial sum projection on $L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))$ given by
$
S_{E_j}f(z)=\sum_{n\in E_j} \widehat{f}(n) z^n
$, where $z\in \mathbb{T}^2$. Then
		\[
		\|f\|_{L^p(\mathbb{T}^2;S_p)} \asymp
		\begin{cases}
		\inf\left\{ \left\|\left(\sum_{j\in\mathbb{Z}^2}g_j g_j^*\right)^{\frac12} \right\|_{L^p(\mathbb{T}^2;S_p)}+\left\|\left(\sum_{j\in\mathbb{Z}^2}h_j^* h_j\right)^{\frac12} \right\|_{L^p(\mathbb{T}^2;S_p)}:S_{E_j}f=g_j+h_j\right\} & p\in(1,2]\\
   \left\| \left(\sum_{j\in\mathbb{Z}^2} (S_{E_j} f)^*(S_{E_j} f)\right)^{\frac12}\right\|_{L^p(\mathbb{T}^2;S_p)}+ \left\| \left(\sum_{j\in\mathbb{Z}^2} (S_{E_j} ) (S_{E_j} f)^*\right)^{\frac12}\right\|_{L^p(\mathbb{T}^2;S_p)} & p\in [2,\infty).
		\end{cases}
   \]
	\end{lemma}
\end{comment}
\begin{comment}
\begin{figure}[ht]
    \centering

    \def0.5\columnwidth{0.25\columnwidth}
    \import{./figures/}{drawing.pdf_tex}
    \caption{Drawing}
    \label{fig:A drawing}
\end{figure}
\end{comment}

With the notations of difference and partition of $\mathbb{Z}^2$, we are ready to state the generalization of Theorem \ref{thm:main} in~$\mathbb{Z}^2$.
\begin{theorem} \label{thm:dimension2}
Given $M=(m_{s,t})_{s,t\in\mathbb{Z}^2}\in \mathcal{B}(\ell^2(\mathbb{Z}^2))$. Suppose $M$ satisfies that
\begin{itemize}
	\item[i)] $\sup_{s,t\in\mathbb{Z}^2} |m_{s,t}|<C_1$
	\item[ii)] For any $k\in\mathbb{N}, s\in\mathbb{Z}^2 $
	there is
	\begin{align}\label{eq:left}
	\left(\sum_{\{t_1:t=(t_1,\pm 2^{k-1})\in E_k\}} |\Delta_{t_1} m_{s,s+t}|  + \sum_{\{t_2:t=(\pm 2^{k-1}, t_2)\in E_k\}} |\Delta_{t_2} m_{s,s+t}| \right) &<C_2, \\
	 \sum_{\{t_1,t_2: t=(t_1,t_2)\in E_k\}} |\Delta_t m_{s,s+t}|&<C_3, \label{eq:leftDouble}
	\end{align}
	and
 \begin{align}\label{eq:right}
	\left(\sum_{\{t_1:t=(t_1,\pm 2^{k-1})\in E_k\}} |\Delta_{t_1} m_{s+t,s}|  + \sum_{\{t_2:t=(\pm 2^{k-1}, t_2)\in E_k\}} |\Delta_{t_2} m_{s+t,s}| \right) &<C_2\\
	 \sum_{\{t_1,t_2: t=(t_1,t_2)\in E_k\}} |\Delta_t m_{s+t,s}|&<C_3, \label{eq:rightDouble}
	 \end{align}
\end{itemize}
Then $M$ is a (completely) bounded Schur multiplier on $S_p(\ell^2(\mathbb{Z}^2))$ for $p\in (1,\infty)$ with the bound $C(\frac{p^2}{p-1})^4$. Here $C_1, C_2$ and $C_3,C$ are some positive constants.
\end{theorem}

 Now we come to the proof of Theorem \ref{thm:dimension2}. Denote $S_{E_j}$ the partial sum projection on $L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))$ given by
$
S_{E_j}f(z)=\sum_{n\in E_j} \widehat{f}(n) z^n
$, where $z\in \mathbb{T}^2$.

By the Lemma \ref{thm:Littwood-P}  and the Schur and Fourier multiplier transference in $\mathbb{Z}^2$ which is defined similar to the one dimensional case, it is sufficient to prove
\begin{equation}\label{eq:SufficientCondition}
\left\| \left( \sum_{j = 0}^{\infty} \left| S_{E_{j }} T_{\widetilde{M}} f\right|^{2} \right)^{\frac{1}{2}} \right\|_{L^{p}(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \leq  C \frac{p^2}{p-1} \left\| \left( \sum_{j = 0}^{\infty} \left| S_{\delta_{j }}  f  \right|^{2} \right)^{\frac{1}{2}} \right\|_{L^{p}(\mathbb{T}^2;S_p(\mathbb{Z}^2))}.
\end{equation}
for $p\geq 2$ and $f(z)\in \pi(\mathcal{B}(\ell^2(\mathbb{Z}^2)))$.
\begin{comment}
To prove the ....
We need the following operator valued Littlewood-Paley theorem in $\mathbb{Z}^2$.

\begin{theorem}
	Suppose $f(z)=$
\end{theorem}
\end{comment}

Notice that \eqref{eq:left} and \eqref{eq:leftDouble} are equivalent to
\begin{align} \label{eq:leftPartialMl}
  \sup_{j\geq 0} \left(  \left\| \sum_{\{n_1:n=(n_1,\pm 2^{j-1})\in E_j\}} |\Delta_{n_1} M_{l}  (n)| \right\|_{\infty}  +\left\| \sum_{\{n_2:n=(\pm 2^{j-1},n_2)\in E_j\}}  |\Delta_{n_2} M_{l} (n )| \right\|_{\infty}  \right) & < C_2,   \\
 \sup_{j\geq 0}\left\|\sum_{\{n_1,n_2: n=(n_1,n_2)\in E_j\}} \left|  \Delta_n M_{l}  (n) \right| \right\|_{\infty}& < C_3.  \label{eq:leftDoublePartial}
\end{align}

To prove \eqref{eq:SufficientCondition}, we may cut $E_j$ into 4 rectangles $E_{j,k}, k=1,\cdots 4$ for $j\geq 1$. Let $I_{j}=[2^{j-1}, 2^j)\cap \mathbb{Z}, J_i=(-2^{j-1},2^j)\cap \mathbb{Z}$, denote
\begin{equation*}
	E_{j,1}=J_j\times I_j,\qquad E_{j,2}=(-I_j)\times J_j, \qquad E_{j,3}= I_j\times (-J_{j}),\qquad E_{j,4}=(-J_j)\times (-I_{j}).
\end{equation*}

\begin{comment}
\begin{figure}[ht]
    \centering
    \def0.5\columnwidth{0.5\columnwidth}
    \import{./figures/}{E2.pdf_tex}
    \caption{Decomposition of $E_j$}
    \label{fig:A drawing2}
\end{figure}
\end{comment}
Thus, we have
\begin{equation}\label{eq:combin}
	S_{E_j}T_{\widetilde{M}} f= \sum_{k=1}^4 S_{E_{j,k}}T_{\widetilde{M}}f.
\end{equation}
Since $E_{j,k}$ can be obtained from $E_{j,1}$ by rotation for $k=2,3,4$. To prove \eqref{eq:SufficientCondition}, it is sufficient to prove
\begin{equation}\label{eq:sufficient2}
\left\| \left( \sum_{j = 0}^{\infty} \left| S_{E_{j,1 }} T_{\widetilde{M}} f\right|^{2} \right)^{\frac{1}{2}} \right\|_{L^{p}(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \leq  C^{\prime} \left(\frac{p^2}{p-1}\right)^2 \left\| \left( \sum_{j = 0}^{\infty} \left| S_{\delta_j}  f  \right|^{2} \right)^{\frac{1}{2}} \right\|_{L^{p}(\mathbb{T}^2;S_p(\mathbb{Z}^2))}
\end{equation}
for $p\geq 2$ and  $f(z)\in \pi(\mathcal{B}(\ell^2(\mathbb{Z}^2)))$. By fundamental theorem of calculus,
\begin{align}\label{eq:abelianSummationDimension2}
	S_{E_{j,1}} T_{\widetilde{M}} f
	= &M_l(-2^{j-1}+1, 2^{j-1}) S_{E_{j,1}}f + \sum_{n_1\in J_j} \Delta_{n_1} M_l(n_1, 2^{j-1}) S_{(n_1,2^j)\times E_j} f \nonumber \\
 &+\sum_{n_2\in E_j} \Delta_{n_2} M_l (-2^{j-1}+1,n_2) S_{J_j\times (n_2,2^j)} f + \sum_{n=(n_1,n_2)\in E_{j,1}} \Delta_n M_l (n_1,n_2) S_{(n_1,2^j)\times (n_2,2^j)}f \nonumber\\
 &=: P^1_j + P^2_j + P^3_j + P^4_j
\end{align}
By the operator inequality  $|\sum_{k=1}^n a_k|^2\leq n \sum_{k=1}^n |a_k|^2$, we have
\begin{equation}\label{eq:Dimension2Part2}
	|S_{E_{j,1}} T_M f|^2 = |P^1_j + P^2_j + P^3_j + P^4_j|^2 \leq 4(|P^1_j|^2+ |P^2_j|^2+|P^3_j|^2+|P^4_j|^2).
\end{equation}
For part $P^1_j$, by assumption $i)$ of Theorem \ref{thm:dimension2}, there is
\begin{equation}\label{eq:partA}
	|P^1_j|^2= | M_l(-2^{j-1+1}, 2^{j-1}) ~S_{E_{j,1}}f|^2 \leq C_1^2 |S_{E_{j,1}}f|^2 \leq C_1^2 |S_{\delta_j} f|^2  .
\end{equation}
For part $P^2_j$, following the similar arguments for \eqref{eq:sumP2} and \eqref{eq:sumP3} in the one dimension case and denote $R_{n_1,j}=(n_1, 2^j)\times E_j$. There is
\begin{align}\label{eq:partB1}
	|P^2_j|^2 &= \left|\sum_{n_1\in J_j} \Delta_{n_1} M_l(n_1, 2^{j-1}) S_{(n_1,2^j)\times E_j} f \right|^2  \nonumber\\
	& \leq \left\|\sum_{n_1\in J_j} |\Delta_{n_1}M_l(n_1,2^{j-1})| \right\|_{\infty} \left( \sum_{n_1\in J_j} \left|S_{R_{n_1,j}} (|\Delta_{n_1} M_l(n_1, 2^{j-1})|^{\frac12}S_{E_j}f) \right|^2 \right).
\end{align}
\begin{comment}
Before we proceed, we need to estimate $\left\|\sum_{n_1\in J_j} |\Delta_{n_1}M_l(n_1,2^{j-1})| \right\|_{\infty}$. By the assumption of \eqref{eq:left} and \eqref{eq:leftDouble},
\begin{align} \label{eq:partB2}
	\left\|\sum_{n_1\in J_i} \left| \Delta_{n_1} M_l(n_1,2^{j-1}) \right| \right\|_{\infty} &= \left\| \sum_{n_1\in J_j} \left| \Delta_{n_1} M_{n_1}(n_1, 2^j-1)- \sum_{n_2\in E_j} \Delta_{n_2}\Delta_{n_1} M_l (n_1, n_2)  \right| \right\|_{\infty}  \nonumber\\
	&\leq \left\| \sum_{n_1\in J_j} \left|\Delta_{n_1} M_l(n_1, 2^j-1) \right|   \right\|_{\infty} + \left\| \sum_{n\in E_{j,1}} \left| \Delta_n M_l (n)    \right|   \right\|_{\infty} \nonumber\\
	&\leq C_2+C_3=: C_4.
\end{align}
\end{comment}
Thus, by \eqref{eq:partB1}
\begin{align}\label{eq:partB3}
\left\| \left(\sum_{j\geq 0}  |P^2_j|^2 \right)^{\frac12} \right\|_{L^p(\mathbb{T}^2; S_p(\mathbb{Z}^2))} &\leq C_4^{\frac12} \left\|  \left(  \sum_{j\geq 0} \sum_{n_1\in J_j} \left| S_{R_{n_1,j}} S_{\delta_j} | \Delta_{n_1} M_l (n_1, 2^{j-1}) |^{\frac12} f   \right|^2     \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))}  \nonumber \\
\leq C_4^{\frac12} \left(\frac{p^2}{p-1}\right) & \left\|  \left(  \sum_{j\geq 0} \sum_{n_1\in J_j} \left|  S_{\delta_j} | \Delta_{n_1} M_l (n_1, 2^{j-1}) |^{\frac12} f   \right|^2     \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \nonumber \\
\leq  C_4^{\frac12} \left(\frac{p^2}{p-1}\right) &\left\|  \left(  \sum_{j\geq 0} (S_{\delta_j})^* \left( \sum_{n_1\in J_j} \left| \Delta_{n_1} M_l(n_1, 2^{j-1}) \right| \right) S_{\delta_j}f    \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \nonumber\\
\leq  C_4 \left(\frac{p^2}{p-1}\right) &\left\|  \left(  \sum_{j\geq 0}  |S_{\delta_j}f|^2  \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))}
\end{align}
For part $P^3_j$, similarly, we have
\begin{equation} \label{eq:partC}
\left\| \left(\sum_{j\geq 0}  |C_j|^2 \right)^{\frac12} \right\|_{L^p(\mathbb{T}^2; S_p(\mathbb{Z}^2))}  \leq    C_4 \left(\frac{p^2}{p-1}\right) \left\|  \left(  \sum_{j\geq 0}  |S_{\delta_j}f|^2  \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))}.
\end{equation}
Now we come to the estimate of part $P^4_j$. Denote $R_{n,j}=(n_1,2^j)\times (n_2,2^j)$. Similarly,
\begin{align}\label{eq:partD}
\left\|  \left( \sum_{j\geq 0}  |P^4_j|^2 \right)^{\frac12} \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))} & \leq C_3^{\frac12}  \left\|  \left(  \sum_{j\geq 0} \sum_{n=(n_1,n_2)\in E_{j,1}} \left|  S_{R_{n,j}} S_{\delta_j} |\Delta_n M_l(n)|^{\frac12} f    \right|^2   \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))}   \nonumber\\
\leq C_3^{\frac12} \left(\frac{p^2}{p-1} \right)^2  & \left\|  \left(  \sum_{j\geq 0} \sum_{n=(n_1,n_2)\in E_{j,1}} \left| S_{\delta_j} |\Delta_n M_l(n)|^{\frac12} f    \right|^2   \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \nonumber \\
\leq C_3 \left(\frac{p^2}{p-1} \right)^2  &\left\| \left(\sum_{j\geq 0} |S_{\delta_j}f|^2  \right)^{\frac12}  \right\|_{L^p(\mathbb{T}^2;S_p(\mathbb{Z}^2))}.
\end{align}

Therefore, by \eqref{eq:combin}, \eqref{eq:abelianSummationDimension2} and \eqref{eq:partB1},\eqref{eq:partB3}, \eqref{eq:partC}, \eqref{eq:partD}, we have
\begin{align}
&\left\| \left( \sum_{j = 0}^{\infty} \left| S_{E_{j,1 }} T_{\widetilde{M}} f\right|^{2} \right)^{\frac{1}{2}} \right\|_{L^{p}(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \leq  2 \left\|  \sum_{j = 0}^{\infty} ( |P^1_j|^2+ |P^2_j|^2 + |P^3_j|^2 + |P^4_j|^2 )  \right\|^{\frac12}_{L^{\frac{p}{2}}(\mathbb{T}^2;S_p(\mathbb{Z}^2))} \nonumber \\
& \leq 2 \left(  \left\| \sum_{j\geq 0} |P^1_j|^2 \right\|_{L^{\frac{p}{2}} (\mathbb{T}^2;S_p(\mathbb{Z}^2))}  + \left\| \sum_{j\geq 0} |P^2_j|^2 \right\|_{L^{\frac{p}{2}} (\mathbb{T}^2;S_p(\mathbb{Z}^2))} +  \left\| \sum_{j\geq 0} |P^3_j|^2 \right\|_{L^{\frac{p}{2}} (\mathbb{T}^2;S_p(\mathbb{Z}^2))} +   \left\| \sum_{j\geq 0} |P^4_j|^2 \right\|_{L^{\frac{p}{2}} (\mathbb{T}^2;S_p(\mathbb{Z}^2))}   \right)^{\frac12} \nonumber \\
&\leq C^{\prime} \left(\frac{p^2}{p-1}\right)^2 \left\| \left( \sum_{j = 0}^{\infty} \left| S_{\delta_{j }} f\right|^{2} \right)^{\frac{1}{2}} \right\|_{L^{p}(\mathbb{T}^2;S_p(\mathbb{Z}^2))}
\end{align}
Thus, \eqref{eq:sufficient2} is proved. Hence, we finish the proof of  theorem \ref{thm:dimension2}.
\medskip

\noindent{ \bf Open Question.}
  Assume    $m$ is a  bounded map on ${\Bbb Z}\times{\Bbb Z}$  such that
 \begin{eqnarray*}
 \sum_s |m(k,j_s)-m(k,j_{s+1})|^2&<&C\\
  \sum_s |m(k_s,j )-m(k_{s+1},j)|^2&<&C
 \end{eqnarray*}
 for all possible increasing sequences $j_s,k_s\in {\Bbb Z}$. Does $M_m$ extend to a (completely) bounded map on  $S_p$ for all $1<p<\infty$?

\begin{remark} \cite{MeXu07}   claimed a positive answer to the question above. But the argument  contains a serious mistake.   The third author tried different methods without success but  found Theorem 1.1 in the process.
\end{remark}

\subsection{Higher dimensional case}
We need some additional notations to deal with $d$-dimensional case. Borrowing the notations from \cite{Hytoenen2016},  we denote by
\[
\mathbb{Z}^{\alpha}:=\{(n_i)_{i:\alpha_i=1}:n_i\in \mathbb{Z} \}
\]
the space $\mathbb{Z}^{|\alpha|}$ for $\alpha\in \{0,1\}^d$ with an indexing of the components derived from $\alpha$. For any $n\in\mathbb{Z}^d$ and $E= I_1\times \cdots \times I_d\subseteq \mathbb{Z}^d$, let
\[
	n_{\alpha}\coloneqq (n_i)_{i:\alpha_i=1}\in \mathbb{Z}^{\alpha}, \quad E_{\alpha}\coloneqq \prod_{i:\alpha_i=1} I_i\subseteq \mathbb{Z}^{\alpha}
\]
be their natural projections onto $\mathbb{Z}^{\alpha}$. In particular, we will use the splittings $n=(n_{\alpha},n_{1-\alpha})\in \mathbb{Z}^{\alpha}\times \mathbb{Z}^{\mathbf{1}-\alpha}$ and $E=E_{\alpha}\times E_{\mathbf{1}-\alpha}$ where $\mathbf{1}=(1,\dots,1)$.  Suppose $s,t\in \mathbb{Z}^d$ and  we  abbreviate the interval notation  $[s,t)\cap \mathbb{Z}^d$ as $[s,t)$.

Similarly, denoted a partition $\mathscr{I}^d \coloneqq \{E_j:j\geq 0\}$ of $\mathbb{Z}^d$, where

\begin{equation}\label{eq:partitionZd}
	E_j=\begin{cases}
		 \{(0,\cdots,0) \}  & j=0,\\
		 \{(n_1,\cdots,n_d)\in\mathbb{Z}^d:2^{j-1}\leq |(n_1,\cdots,n_d)|_{\infty}<2^j \} & j\geq 1.
	\end{cases}
\end{equation}

Each $E_j$ can be further decomposed into $2^d(2^d-1)$ subsets and each of the subsets can be obtained by translation of the cube $F_j=[2^{j-1},2^j)\times \cdots \times[2^{j-1},2^j)$. Denote $s^{(j)}=(2^{j-1},\cdots, 2^{j-1}), j\geq 1, $ then $F_j$ can be denoted as $[{t}^{(j)},{t}^{(j+1)})$.
Following the similar procedures in the two dimensional case and using  the discrete fundamental theorem formula which takes the following form
\begin{align}\label{eq:fundamentalCalculus}
	\chi_{[s,t)} (n) M(n) &= \chi_{[s,t)} \sum_{\alpha\in \{0,1\}^d} \sum_{k_\alpha\in [s,n)_{\alpha}} \Delta^{\alpha} M(s_{\mathbf{1}-\alpha},k_{\alpha}) \nonumber \\
	&= \sum_{\alpha\in \{0,1\}^d} \sum_{k_{\alpha}\in [s,t)_{\alpha}} \chi_{[k,t)_{\alpha}\times [s,t)_{\mathbf{1}-\alpha}}(n) \Delta^{\alpha}M(s_{\mathbf{1}-\alpha},k_{\alpha})
\end{align}
for the function $M:\mathbb{Z}^d\to \mathbb{C}$, $s,t\in\mathbb{Z}^d$, we obtain the following theorem. The details are left to the interested readers.
\begin{theorem}\label{higerd}
	Given $M=(m_{s,t})_{s,t\in\mathbb{Z}^d}\in \mathcal{B}(\ell^2(\mathbb{Z}^d))$. Suppose $M$ satisfies that
	\begin{itemize}
	\item[i)] $\sup_{s,t\in\mathbb{Z}^d} |m_{s,t}|<C$,
		\item[ii)] For any $k\in\mathbb{N}$, $s\in \mathbb{Z}^d, \alpha\in \{0,1\}^d$ and $\alpha\neq \mathbf{0}$, there is
		\begin{equation}
		\sum_{\{t_{\alpha}:t=(t_{\alpha}, t^{(k)}_{\mathbf{1}-\alpha})\in E_k \} } |\Delta^{\alpha} m_{s,s+t}|<C,\quad \sum_{\{t_{\alpha}:t=(t_{\alpha}, t^{(k)}_{\mathbf{1}-\alpha})\in E_k \} } |\Delta^{\alpha} m_{s+t,s}|<C ,
		\end{equation}
	\end{itemize}
	where $C$ is some positive constant. Then $M$ is a (completely) bounded Schur multiplier on $S_p(\ell^2(\mathbb{Z}^d))$ for $p\in (1,\infty)$ with the bound $C_d (\frac{p^2}{p-1})^{d+2}$ where $C_d$ is some constant only dependent on the dimension $d$.
\end{theorem}

\section{Schur-Marcinkiewicz Multiplier theorems in continuous cases}
We would like to extend the Theorem \ref{thm:main} and Theorem \ref{higerd} to continuous case via the method of approximation in \cite{Lafforgue2011}.
Let $S_p(\mathbb{R}^n)$ denote the Schatten $p$-class acting on $L^2(\mathbb{R}^n)$.

\begin{theorem}
For $p\in (1,\infty)$, consider the Schur multiplier $S_M$ on $S_p(\mathbb{R})$ with symbol $M (\cdot,\cdot)\in L^{\infty} (\mathbb{R}^{2})$ whose partial derivatives are continuous in every dyadic interval set $(-2^{j+1},-2^j)\bigcup (2^j, 2^{j+1})$ for $j~\in~\mathbb{Z}$. If $M$ satisfies

\begin{equation} \label{eq: con 1}
\sup_{j \in \mathbb{Z}} \sup_{y \in \mathbb{R}} \left\{ \int_{- 2^{j + 1}}^{- 2^{j}} \left| \partial_{1}  M   (y + t, y)  \right| \ d t + \int_{2^{j}}^{2^{j + 1}} \left|  \partial_{1}  M  (y + t, y) \right| \ d t \right\} \leq A
\end{equation}
and
\begin{equation} \label{eq: con 2}
\sup_{j \in \mathbb{Z}} \sup_{x \in \mathbb{R}} \left\{ \int_{- 2^{j + 1}}^{- 2^{j}} \left| \partial_{2}  M    (x, x + t) \right| \ d t + \int_{2^{j}}^{2^{j + 1}} \left|  \partial_{2} M  (x, x + t) \right| \ d t \right\} \leq A.
\end{equation}
Then, the Schur multiplier $S_{M} : S_p(\mathbb{R})\to S_p(\mathbb{R})$ is bounded with $\| S_{M} \|\leq C \max \left\{ p^{3}, \frac{1}{(p - 1)^{3}} \right\}$.
\end{theorem}
\begin{proof}
Let $k \in \mathbb{N}$. Consider the measure space $\mathbb{R}^{2}$ with the Lebesgue measure. Let $\mathcal{B}_{k}$ be the $\sigma$-algebra generated by the half-open parallelogram $D_{k, a, b}$ with vertices $\left(\frac{a}{2^{k}}, \frac{b}{2^{k}} \right)$, $\left(\frac{a + 1}{2^{k}}, \frac{b + 1}{2^{k}} \right)$, $\left( \frac{a}{2^{k}}, \frac{b + 1}{2^{k}} \right)$, $\left( \frac{a + 1}{2^{k}}, \frac{b + 2}{2^{k}} \right)$, where $a$, $b \in \mathbb{Z}$.

\begin{center}
  \begin{tikzpicture}
    \node (n) [draw, minimum width=2cm, minimum height=2cm, xslant= - 1.1] {};
    \draw     (n.north west) node[above] {$\left( \frac{a}{2^{k}}, \frac{b}{2^{k}} \right)$}
              (n.north east)  node[above] {$\left( \frac{a}{2^{k}}, \frac{b + 1}{2^{k}} \right)$}
              (n.south west) node[below] {$\left( \frac{a + 1}{2^{k}}, \frac{b + 1}{2^{k}} \right)$}
              (n.south east) node[below] {$\left( \frac{a + 1}{2^{k}}, \frac{b + 2}{2^{k}} \right)$};
  \end{tikzpicture}
\end{center}
let $\mathscr{D}_{k}$ be the collection of all half-open parallelogram of the above form. Note that  $(\mathcal{B}_{k})_{k = 1}^{\infty}$ forms a filtration for $\mathbb{R}^{2}$ equipped with the Lebesgue measure. Given a Lebesgue measurable function $M$ on $\mathbb{R}^{2}$, the conditional expectation $M_k$ of $M$ with respect to the sub-$\sigma$-algebra $\mathcal{B}_{k}$ is given by
\[
M_{k} (x) = \sum_{Q \in \mathscr{D}_{k}} \text{Avg}_{Q} (M) \chi_{Q} (x).
\]
Note that $\left\| M_{k} \right\|_{M[S^{p} (L^{2} (\mathcal{B}_{k}))]} = \left\| \widetilde{M_{k}} \right\|_{M [S^{p} (\ell^{2} (\mathbb{Z}))]}$, where $\widetilde{M_{k}} = [ m_{k} (s, t)]_{s, t \in \mathbb{Z}}$ and
\[
m_{k} (s, t) = \text{Avg}_{ D_{k, s, t}} (M).
\]

First, we show that $\widetilde{M_{k}}$ satisfies the assumptions for the discrete case. For each $j \in \mathbb{N}$ and $s \in \mathbb{Z}$,
\begin{align} \label{eq:continue}
&\sum_{\ell = 0}^{2^{j} - 2} \left| m_{k} \left( s, s + 2^{j} + \ell + 1 \right) - m_{k} \left( s, s + 2^{j} + \ell \right) \right|  \nonumber\\
&= 2^{2 k}\sum_{\ell = 0}^{2^{j} - 2} \left| \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j} + \ell + 1}{2^{k}}}^{{\frac{2^{j} + \ell + 2}{2^{k}}}} M \left( y, y + x \right) \ d x \ d y- \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j} + \ell}{2^{k}}}^{{\frac{2^{j} + \ell + 1}{2^{k}}}} M \left( y, y
+ x \right) \ d x \ d y \right|  \nonumber\\
&= 2^{2 k}\sum_{\ell = 0}^{2^{j} - 2} \left| \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j} + \ell}{2^{k}}}^{{\frac{2^{j} + \ell + 1}{2^{k}}}} M ( y, y + x + \frac{1}{2^{k}} ) \ d x \ d y - \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j} + \ell}{2^{k}}}^{{\frac{2^{j} + \ell + 1}{2^{k}}}} M \left( y, y + x \right) \ d x \ d y \right|  \nonumber\\
&= 2^{2 k}\sum_{\ell = 0}^{2^{j} - 2} \left| \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j} + \ell}{2^{k}}}^{{\frac{2^{j} + \ell + 1}{2^{k}}}} \int_{y + x}^{y + x + \frac{1}{2^{k}}}  \partial_{2} M \left( y, t \right) \ d t \ d x \ d y \right|  \nonumber\\
&\leq 2^{2 k}\sum_{\ell = 0}^{2^{j} - 2} \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j} + \ell}{2^{k}}}^{{\frac{2^{j} + \ell + 1}{2^{k}}}} \int_{y + x}^{y + x + \frac{1}{2^{k}}} \left| \partial_{2} M \left( y, t \right) \right| d t \ d x \ d y  \nonumber\\
&= 2^{2 k} \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{\frac{2^{j}}{2^{k}}}^{{\frac{2^{j + 1} - 1}{2^{k}}}} \int_{y + x}^{y + x + \frac{1}{2^{k}}} \left|  \partial_{2} M \left( y, t \right) \right| d t \ d x \ d y  \nonumber\\
&= 2^{2 k} \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{y + \frac{2^{j}}{2^{k}}}^{{y + \frac{2^{j + 1}}{2^{k}}}} \int_{\frac{2^{j}}{2^{k}}}^{{\frac{2^{j + 1} - 1}{2^{k}}}} \chi_{\left( t - \frac{1}{2^{k}}, t \right)} (x) \ d x \left| \left[ \partial_{2} (M) \right] \left( y, t \right) \right|  d t \ d y  \nonumber\\
&\leq 2^{k} \int_{\frac{s}{2^{k}}}^{\frac{s + 1}{2^{k}}} \int_{y + \frac{2^{j}}{2^{k}}}^{{y + \frac{2^{j + 1}}{2^{k}}}} \left|  \partial_{2} M \left( y, t \right) \right|  d t \ d y
\leq \sup_{y \in \mathbb{R}} \int_{\frac{2^{j}}{2^{k}}}^{{\frac{2^{j + 1}}{2^{k}}}} \left| \left[ \partial_{2} (M) \right] \left( y, y + t \right) \right|  d t \leq A.
\end{align}
The last inequality in \eqref{eq:continue} is from the assumption \eqref{eq: con 2}.Using the same method, we also obtain that for each $j \in \mathbb{N}$ and $s \in \mathbb{Z}$,
\[
\sum_{\ell = 0}^{2^{j} - 2} \left| m_{k} \left( s, s - 2^{j} - \ell - 1 \right) - m_{k} \left( s, s - 2^{j} - \ell \right) \right| \leq A.
\]
Similarly, by \eqref{eq: con 1}, we also have that for each $j \in \mathbb{N}$ and $s \in  \mathbb{Z}$,
\[
\sum_{\ell = 0}^{2^{j} - 2} \left| m_{k} \left( s + 2^{j} + \ell + 1, s \right) - m_{k} \left( s + 2^{j} + \ell, s \right) \right| \leq A
\]
and
\[
\sum_{\ell = 0}^{2^{j} - 2} \left| m_{k} \left( s - 2^{j} - \ell - 1, s \right) - m_{k} \left( s - 2^{j} - \ell, s \right) \right| \leq A.
\]
By the Theorem \ref{thm:main}, $\left\| M_{k} \right\|_{M[S^{p} (L^{2} (\mathcal{B}_{k}))]} = \left\| \widetilde{M_{k}} \right\|_{M [S^{p} (\ell^{2} (\mathbb{Z}))]} \leq C \max \left\{ p^{3}, \frac{1}{(p - 1)^{3}} \right\}$
It then follows from Lemma 1.11 in \cite{Lafforgue2011} that
\[
\left\| M \right\|_{M[S^{p} (L^{2} (\mathbb{R}^{2}))]}= \lim_{k \to \infty} \left\| M_{k} \right\|_{M[S^{p} (L^{2} (\mathcal{B}_{k}))]} \leq C \max \left\{ p^{3}, \frac{1}{(p - 1)^{3}} \right\}.
\]\end{proof}
Similarly, we can extend Theorem \ref{higerd} to the continuous case using approximation property in \cite{Lafforgue2011} and obtain the following theorem.

\begin{theorem}
Denote  $F_{j} := \{ t \in \mathbb{R}^{d} : 2^{j} < | t |_{\infty} \leq 2^{j + 1} \}$ for $j\in\mathbb{Z}$.  For $p\in (1,\infty)$, consider the Schur multiplier $M$ on $S_p (\mathbb{R}^{d}) $ with symbol $M \in L^{\infty} (\mathbb{R}^{2d})$ whose partial derivatives are continuous up to the boundary of $F_j$ for all $j\in\mathbb{Z}$. Assume that $M$ satisfies
\begin{align} \label{eq:rightCondition}
&\sup_{j \in \mathbb{N}} \sup_{s \in \mathbb{R}^d} \int_{(t_{\alpha}, t_{1-\alpha}^{(k)}) \in F_{j}} \left|  \partial_{\alpha} M ( s,s  + t ) \right| dt_\alpha  \leq A\\
	&\sup_{j \in \mathbb{N}} \sup_{s \in \mathbb{R}} \int_{(t_{\alpha}, t_{1-\alpha}^{(k)}) \in F_{j}} \left|  \partial_{\alpha} M ( s+t, s   ) \right| dt_{\alpha}\leq A
\end{align}

Then, the Schur multiplier $S_{M} : S_p(\mathbb{R}^d)\to S_p(\mathbb{R}^d)$ is bounded with $\| S_{M} \|\leq C \max \left\{ p^{d+2}, \frac{1}{(p - 1)^{d+2}} \right\}$.

\end{theorem}

\begin{comment}

\begin{theorem}
	continuous case dimension 2, to be added...
\end{theorem}

\begin{theorem}
higher dimension n, to be added...
\end{theorem}

\end{comment}
\bibliographystyle{abbrv}