Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Training Transformers for Information Security Tasks: A Case Study on
  Malicious URL Prediction   Machine Learning (ML) for information security (InfoSec) utilizes distinct
data types and formats which require different treatments during
optimization/training on raw data. In this paper, we implement a
malicious/benign URL predictor based on a transformer architecture that is
trained from scratch. We show that in contrast to conventional natural language
processing (NLP) transformers, this model requires a different training
approach to work well. Specifically, we show that 1) pre-training on a massive
corpus of unlabeled URL data for an auto-regressive task does not readily
transfer to malicious/benign prediction but 2) that using an auxiliary
auto-regressive loss improves performance when training from scratch. We
introduce a method for mixed objective optimization, which dynamically balances
contributions from both loss terms so that neither one of them dominates. We
show that this method yields performance comparable to that of several
top-performing benchmark classifiers.

12
The categories of this paper are: nucl-th
17
The categories of this paper are: q-fin
26
The categories of this paper are: bayes-an
29
The categories of this paper are: funct-an
23
The categories of this paper are: solv-int
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
28
The categories of this paper are: alg-geom
32
The categories of this paper are: atom-ph
16
The categories of this paper are: stat
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
19
The categories of this paper are: eess
10
The categories of this paper are: q-bio
27
The categories of this paper are: comp-gas
37
The categories of this paper are: supr-con