Sequential Recommendation with Multiple Contrast Signals Sequential Recommendation with Multiple Contrast Signals

3.2.1 Revisiting Typical Training Method of Sequential Recommendation. Currently, most studies about sequential recommendation follow a supervised learning paradigm, where an interacted item is the prediction target, and its historical sequence serves as the input. Then, a BPR pairwise ranking loss as Equation (2) is generally optimized to train the sequential recommendation model. Here we show that such training strategy is actually a specialized contrastive learning task for two reasons, both conceptually and mathematically.

Firstly, different from supervised tasks in other domains, where the label information totally comes from external knowledge, the inputs and labels in sequential recommendation inherently exist in the original user interaction sequences. Current studies manually decouple the complete sequence into input historical sequences and target items to discriminate between them. Given the historical sequence, the target item is driven to get a higher ranking score compared to other items. This is conceptually consistent with the setting of contrastive learning: the historical sequence and the target item are similar samples directly extracted from the data, while other items are dissimilar instances given the historical sequence.

Secondly, the BPR pairwise ranking loss, i.e., Equation (2), can be proven a variant of contrastive loss as follows:

Therefore, the commonly adopted BPR loss in sequential recommendation is intrinsically related to the InfoNCE loss in contrastive learning. This typical training method actually discriminates between the representations of historical sequences and target items by optimizing a variant of contrastive loss.

3.2.2 Context-Target Contrastive Loss. Given the analyses above, the typical training method of sequential recommendation can be seen to discriminate between historical sequences and target items, named context-target contrast in this paper. Instead of directly optimizing BPR loss as the common practice, we further extend BPR to a general contrastive loss. Here we loosen the constraints summarized above and devise the following context-target contrastive (CTC) loss:

Compared to the original BPR loss, CTC loss naturally supports multiple negative samples. Note that it is also possible to derive an enhanced version of BPR loss when there are multiple negative samples, named BPR+:

In our ContraRec framework, we adopt this CTC loss to learn the context-target contrast task. We will show its superiority compared to BPR and its enhanced version (BPR+) in Section 5.2, especially when the number of negative samples grows.

3.3.1 Sequence Augmentation. The sequence augmentation component $ Aug(\cdot) $ applies random augmentation to the original sequence. For each input historical sequence $ S_t $, we generate two randomly augmented sequences $ \tilde{S_t}^a=Aug(S_t, seed_1) $ and $ \tilde{S_t}^b=Aug(S_t, seed_2) $. Here $ seed_1 $ and $ seed_2 $ are two random seeds that determine the concrete augmentation method (e.g., mask, reorder) and the effect of augmentation (e.g., masked positions). Given the augmentation set, $ Aug(\cdot) $ first randomly chooses a specific augmentation method with equal probability, and then applies it on the input sequence. The design of augmentation methods in the augmentation set may relate to concrete application scenarios and recommendation models. In this work, we present two augmentation methods as examples, which will be detailed in Section 3.5.

3.3.2 Sequence Encoder. The sequence encoder component $ f(\cdot) $ is responsible for encoding the input sequence $ S_t $ into a low-dimensional representation $ f(S_t)\in \mathbb {R}^d $ ($ d $ is the dimension of the representation space). Note that the sequence encoder here is shared with that in the context-target contrastive learning task, i.e., there is only one set of parameters for the sequence encoder in our ContraRec framework. This makes the knowledge learned from different tasks able to benefit each other. As for the concrete architecture of the sequence encoder, there is no specific restriction. In practice, most deep learning based sequence modeling methods cater to the requirement, such as RNN, CNN, Transformer, and so on. This makes ContraRec a general framework, and various deep sequential recommendation models (e.g., GRU4Rec, Caser, BERT4Rec) can be integrated as the sequence encoder.

3.3.3 Context-Context Contrastive Loss. The context-context contrastive loss $ \mathcal {L}_{CCC} $ aims to encourage the representations of similar sequences to be close to each other. Let us denote the original training mini-batch as $ \mathcal {B}=\lbrace (S_1,i_1), (S_2,i_2),\ldots \rbrace $, whose element contains a target item $ i_t $ and corresponding historical sequence $ S_t $. After the sequence augmentation component, we get an augmented sequence set $ \mathcal {A}=\lbrace \tilde{S_1}^a, \tilde{S_1}^b, \tilde{S_2}^a, \tilde{S_2}^b,\ldots \rbrace $ with size $ 2|\mathcal {B}| $, where each original sequence yields two different views. Then we aim to find similar and dissimilar sequences within the augmented sequence set $ \mathcal {A} $ (in-batch comparison).

For one thing, assuming the sequence augmentation can be seen as an intent-invariant transformation, the two sequences derived from the same historical sequence (e.g., $ \tilde{S_1}^a $ and $ \tilde{S_1}^b $) should have similar representations. For another, sequences derived from different historical sequences may also reflect similar user intents if their target items are the same (e.g., $ \tilde{S_1}^a $ and $ \tilde{S_2}^a $ assuming $ i_1=i_2 $). Notice that for both circumstances, the identity of the target item¹ can be utilized as a sign of similar sequences. The augmented sequences also share the same target item because their original historical sequences are identical. Thus, we define the set of similar sequences of $ \tilde{S_t} $ in $ \mathcal {A} $ as $ T(\tilde{S_t})=\lbrace \tilde{S_t^{\prime }}\in \mathcal {A}~|~i_t=i_t^{\prime }\rbrace $. This set includes not only the paired augmented sequence of $ \tilde{S_t} $, but also other augmented sequences that derived from different interaction sequences with the same target item (if exists). As a result, for each element $ \tilde{S_t}\in \mathcal {A} $, there is possibly more than one positive instance in $ \mathcal {A} $.

The common InfoNCE loss only supports one positive instance given an input instance. Thus, we extend InfoNCE loss and devise the following context-context contrastive loss $ \mathcal {L}_{CCC} $ to encourage the representation of a given sequence $ \tilde{S_t}\in \mathcal {A} $ to be close to the representations of all its similar sequences $ \tilde{S_t^{\prime }}\in T(\tilde{S_t}) $:

3.5.1 Mask. In practice, users’ intents are seldom dominated by a single interaction but generally remain stable over a period of time. A user may interact with a series of similar items with a specific intention. Based on this consideration, we propose to use random mask as a sequence augmentation method, which randomly masks a proportion of input tokens to a special mask token. This technique is also common in natural language processing tasks to avoid overfitting [2, 6, 55]. Specifically, the augmentation procedure can be formulated as follows:

Given the historical sequence $ S_t $ (the length denoted as $ N $), we first sample a proportion $ p_1 $ from a Beta distribution³ and determine the number of positions $ n_1 $ we want to mask. Then, we randomly choose $ n_1 $ positions and set corresponding items in the sequence to the mask token.

3.5.2 Reorder. Intuitively, the order of user interactions in most real-world scenarios is in a flexible manner [45, 52]. Users may purchase a set of items simultaneously and the inner order does not matter. Besides, recent self-attention based sequential recommendation models gain significant performance improvements [24, 44] but care less about the temporal order compared to RNN, which implies that the sequence order may be not that important. Therefore, we propose to augment historical sequences based on partial permutation:

Both of the methods introduced above yield an augmented sequence $ \tilde{S_t} $ with the same length $ N $ as the input sequence $ S_t $. It is noteworthy that the sequence augmentation component is not restricted to the proposed two methods. We leave the investigation of other augmentation methods as future work to center our contribution to the overall contrastive learning framework.

3.6.1 Alternative Training Methods of Sequential Recommendation. Recent work [44, 58, 65] also begins to explore other self-supervision signals and alternative training strategies (e.g., masked item prediction, segment prediction, see Section 6 for more details). However, they mainly follow a two-stage learning paradigm, which first pre-train with other self-supervised tasks and fine-tune on the traditional next-item recommendation task. Neither of them investigates the connection between contrastive learning and next-item recommendation with BPR pairwise ranking loss. Differently, ContraRec makes the training of sequential recommendation a holistic contrastive learning framework together with the proposed context-context contrast signal. Besides, the self-supervised tasks in previous studies are mainly based on specific sequence encoders (e.g., Transformer block), while ContraRec does not introduce any extra parameters and serves as a general framework for various sequential recommendation models.

3.6.2 Contrastive Learning in Other Domains. Contrastive learning has achieved great success recently in domains like computer vision (CV) [4, 12, 37] and natural language processing (NLP) [9, 35, 55]. Take CV as an example, related studies usually transform input images into different views and utilize the InfoNCE loss to distinguish whether the augmented images come from the same input. However, in sequential recommendation, it is still not clear whether the data augmentation could benefit the representation learning and how to augment historical sequences without changing the original user intents. Hence, we devise two sequence augmentation methods in ContraRec to validate the effectiveness of data augmentation. Besides, considering the characteristics of sequential recommendation, the proposed $ \mathcal {L}_{CCC} $ also encourages sequences with the same target item to have similar representations, which extends the common InfoNCE loss to support multiple positive pairs. Notice that if there is no historical sequence with the same target item in the mini-batch, $ T(\tilde{S_t}) $ has only one element (the paired augmented sequence of $ \tilde{S_t} $) and $ \mathcal {L}_{CCC} $ degrades to the commonly adopted InfoNCE loss in other domains.

4.1.1 Datasets. We conduct extensive experiments on three public datasets in real-world recommendation scenarios:

• Beauty ⁴: This is one of the series of product review datasets crawled from Amazon [16]. The data is split into separate datasets by the top-level product category.
  
• Yelp-2018 ⁵: This is a popular dataset for business recommendation, including restaurants, bars and so on. We use the transaction records after Jan. 1st, 2018 following previous work [53].
  
• Gowalla ⁶: This is the check-in dataset [32] obtained from Gowalla, where users share their locations by checking-in.
  

To keep consistent with the default “5-core” setting in Amazon datasets, we preprocess the Yelp-2018 and Gowalla dataset to ensure each user and item has at least five associated interactions. The statistics of datasets after preprocessing are summarized in Table 2.

4.1.2 Evaluation Protocols. We adopt the leave-one-out strategy to evaluate model performance, which is widely used in previous work [7, 49, 65]. For each interaction sequence, we use the most recent interaction for testing, the second recent interaction for validation, and the remaining interactions for training. In the meantime, considering that the Gowalla dataset contains repeat interactions for each user,⁷ it is possible that the target item in the validation/test dataset has been seen in the training set. To avoid correlation between the three sets of data, we filter the validation and test data of Gowalla such that the target item is the first instance that the user has not been visited before (around 55% of the users meet the requirement). Some initial experiments were done without the filtering where the results were also positive, but in order to focus on avoiding the potential correlations between the data, only results from the filtered dataset will be presented.

To speed up model evaluation, we randomly sample 1,000 items as negative items, and this setting is shown to be close to the non-sampling version [28, 30]. We employ Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) [23] as evaluation metrics. HR@k measures whether the target item appears in the top-k recommendation list, while NDCG@k further concerns about its position in the ranking list. Let $ g_u\in [1,1001] $ denote the rank of the target item for each user $ u $, then HR@k and NDCG@k can be defined as follows under our experimental settings:

4.1.3 Baselines. We compare our method with various representative works, including state-of-the-art sequential recommendation models and recent studies based on self-supervised learning.

• FPMC [42]: This method combines matrix factorization and Markov Chains with a user-specific transition matrix.
  
• GRU4Rec [19]: This method utilizes GRU (Gated Recurrent Unit, a variant of RNN) [5] to model sequential interactions.
  
• Caser [45]: This is a CNN-based method that captures high-order Markov Chains by applying horizontal and vertical convolutional operations on interaction sequences.
  
• SASRec [24]: This method utilizes self-attention [47] to exploit the mutual influence between historical interactions.
  
• BERT4Rec [44]: This is a state-of-the-art sequential recommendation model that uses bidirectional self-attention [47] to derive sequence representations.
  
• S$ ^3 $Rec [65]: It designs four pretext tasks for context-aware recommendation and then fine-tunes on the next-item recommendation task, which is a state-of-the-art method based on self-supervised learning. Due to the fact that there is no feature information in our setting, we only use the masked item prediction and segment prediction tasks in S$ ^3 $Rec for fairness.
  
• CLRec [64]: This method uses the contrastive loss to optimize an attention-based sequential recommendation model to improve both effectiveness and fairness.
  
• CL4SRec [58]: This is a recent study that also investigates the contrast signal between augmented historical sequences based on contrastive learning. But it only takes sequences augmented from the same input as similar instances and uses a simple softmax loss.
  

4.1.4 Implementation Details. We implement our method with PyTorch and the codes are publicly available to facilitate reproducibility.⁸ We use Adam [26] as the optimizer due to its success in recent deep learning methods. Early stop is adopted if NDCG@5 on the validation dataset continues to drop for 10 epochs. We consider a maximum of 50 recent interactions as the historical sequence.

We use the transformer layer in BERT4Rec as the default sequence encoder in our ContraRec because it has been shown to be effective in most scenarios. We will demonstrate the performance of ContraRec when integrating different base sequence encoders in Section 4.4. As for hyper-parameters of ContraRec, the coefficient of $ \mathcal {L}_{CCC} $ is tuned within [0, 0.01, 0.1, 1, 5, 10]; the temperatures $ \tau _1 $ and $ \tau _2 $ are tuned from 0.1 to 1 with step 0.1 respectively; the batch size $ |\mathcal {B}| $ is tuned between $ [256, 512, 1024, 2048, 4096] $. With regard to the number of negative items in CTC loss, we set $ K=1 $ by default for fair comparisons because baseline sequential recommendation models generally use one negative sample during training. Besides, we will also report the results when $ K=64 $ for ContraRec, denoted as ContraRec(multi). All the parameters are normally initialized with 0 mean and 0.01 standard deviation.

4.2.1 Effectiveness of ContraRec. Firstly, self-supervised learning based methods are generally better than traditional sequential recommendation models, and CL4SRec becomes the strongest baseline. CL4SRec improves the performance of sequential recommendation models through encouraging the representation similarity between augmented sequences, which shows the usefulness of exploiting other contrast signals in sequential recommendation. Further, the proposed ContraRec and ContraRec(multi) achieve the best performance on all the datasets.

Compared to traditional sequential recommendation models, the main difference of ContraRec is optimizing the context-context contrastive loss in addition. The consistent improvements demonstrate that the context-context contrast signal between similar historical sequences indeed helps the model obtain better sequence representations and hence benefits the next-item recommendation task. Compared to S$ ^3 $Rec and CLRec, the superiority of ContraRec shows that the sequence-augmentation based contrastive task is more beneficial to the representation learning in sequential recommendation. Compared to CL4SRec, we use the InfoNCE loss instead of the simple softmax loss, which is shown to be capable of better addressing the exposure bias in recommendation [64]. We also do not restrict the similar sequences to those augmented from the same input sequence, but incorporate sequences with the same target item inspired by the idea of collaborative filtering. This helps the proposed method achieve the best performance consistently.

4.2.2 Efficiency Analyses. Table 3 also shows the efficiency of different methods. To measure the running time, all the experiments are conducted with the same batch size of 256 and on the same machine (Intel Xeon 8-Core CPU of 2.2GHz and single NVIDIA GeForce RTX 2080Ti GPU) for fair compar ison. We notice that, in general, more recent models require more time for a single iteration (time/iter), while the convergence speeds (#iter) of different methods are usually diverse, leading to varying training time in total (total time). On the one hand, regarding the training time for a single iteration, our method is inevitably slower (no more 2$ \times $) than traditional sequential recommendation models because we need to encode additional two augmented sequences in each batch. Similarly, the recent contrastive method CL4SRec also needs more time for a single iteration. On the other hand, ContraRec converges very fast and only requires about 50 iterations to achieve stable performance, while CL4SRec usually needs more than 100 iterations. The reason is that our ContraRec leverages multiple contrast signals and uses the temperature-scaled contrastive loss to help the model focus more on hard negatives. As a result, the total training time of ContraRec is similar with traditional sequential methods (ContraRec and SASRec both need around 9.3h to achieve the optimal performance on the largest Gowalla dataset), which does not introduce

significantly more time costs. Considering the remarkable improvements brought by ContraRec, we believe the performance gains somewhat justify the runtime costs.

4.2.3 Effect of More Negative Samples. Beyond the consistent improvements of ContraRec, we find ContraRec(multi) generally achieves higher performances compared to ContraRec. This indicates the usefulness of more negative samples during training, which brings richer contrast signals and further enhances the model performance. It is more possible to get hard negative items when the number of negative samples increases, hence providing a better estimation of the updating gradient. For general sequential recommendation models, although it is also feasible to use more than one negative sample with $ \mathcal {L}_{BPR+} $ (i.e., Equation (6)), we will experimentally show in Section 5.2 that the proposed $ \mathcal {L}_{CTC} $ benefits more from multiple negative samples. At the same time, experiments in the following sections will adopt the version when $ K=1 $ (i.e., ContraRec instead of ContraRec(multi)) for fair comparisons with other methods, unless noted otherwise.

4.3.1 Effect of Sequence Augmentation. It is noteworthy that the Augment version usually performs a little better than the base sequential recommendation model BERT4Rec. Compared to the base sequential model, the only difference of the Augment version is randomly augmenting the input historical sequence during training. If the augmentation methods actually change the original user intents, the performance of the Augment version is expected to somewhat suffer a loss. On the contrary, the consistent performance improvements of the Augment version imply the rationality of the proposed sequence augmentation methods, namely mask and reorder. On the one hand, user intents are generally stable over a period of time, which will not be greatly influenced if some interactions are missing. On the other hand, user behaviors in many real-world scenarios are generally flexible and do not follow a rigid order.

4.3.2 Comparison of Training Strategy. Different from the joint learning framework of ContraRec, previous work [4, 65] usually adopts the two-stage learning strategy, which first pre-trains model parameters with pretext tasks and then fine-tunes on the main task. Here we treat context-context contrast ($ \mathcal {L}_{CCC} $) as the pretext task and context-target contrast ($ \mathcal {L}_{CTC} $) as the main task. From Table 4, it can be found that ContraRec-Pre also gets consistently better results compared to the base sequential recommendation model. This directly validates the usefulness of the proposed context-context contrastive learning task because it is only used for pre-training. The fine-tuning stage is the same as the training procedure of the base sequential recommendation model. Meanwhile, ContraRec-Pre is still inferior to ContraRec all the time. This suggests that it is better to adopt a holistic contrastive learning framework with joint learning but not just utilize the context-context contrast signal for pre-training. The two kinds of contrastive learning tasks ($ \mathcal {L}_{CTC} $ and $ \mathcal {L}_{CCC} $) potentially benefit each other during training, which is more suitable to be unified together.

4.3.3 Effect of Multiple Positive Pairs. Experiments demonstrate that ContraRec-SP suffers consistent performance losses, which implies the usefulness of multiple positive pairs in context-context contrast. This also differs our ContraRec from CL4SRec and contrastive learning studies in other domains. Although sometimes historical sequences with the same target item might reflect dissimilar user preferences (attracted by different aspects of the item), there are also many cases where the same target item indeed reflects similar user intents. As a result, pushing up the similarity of their representations helps to enhance the capacity of the sequence encoder. Notice that the performance drop of ContraRec-SP is not that large. One possible reason is that some mini-batches may not contain historical sequences with the same target item because there are numerous items in the dataset. Even though, we argue this is a unique supervision signal in sequential recommendation that should be taken into consideration, which enriches the context-context contrast signal and further boosts the recommendation performance.

4.3.4 Effect of Additional Projection Head. Different from the observation in other domains, we find adding a projection head hurts the overall performance of ContraRec. Although ContraRec-AP still outperforms the base sequential recommendation model, it is inferior to ContraRec all the time. This may result from the ranker function in sequential recommendation, which generally takes a simple form like dot product. Unlike models in computer vision that needs another linear classification layer after getting the image representation, sequential recommendation directly recommends items close to the sequence representation. Hence, it may be better to drive the representations of similar sequences to be close to each other in the original representation space.

5.3.1 Coefficient of the Context-Context Contrastive Loss. The coefficient $ \gamma $ in the final objective function Equation (9) controls the importance of the context-context contrastive learning task. Figure 6 shows the NDCG@10 of ContraRec with different base sequential recommendation models when the coefficient ranges within [0, 0.01, 0.1, 1, 5, 10]. First, it can be observed that the context-context contrastive learning task indeed benefits the recommendation performance. It is useful to encourage sequences after augmentation as well as sequences with the same target item to have similar representations. Although the typical context-target contrastive task can also achieve the goal that multiple sequences with the same target learn similar embeddings, the CCC loss further addresses this explicitly. Besides, the knowledge to learn augmentation-invariant representations in the CCC loss can hardly be captured by the traditional learning paradigm. Compared to the situation when only optimizing $ \mathcal {L}_{CTC} $ ($ \gamma =0 $), ContraRec achieves better results under most settings of $ \gamma $ on all the datasets. Besides, the overall trend increases first and then decreases. Notice that $ \gamma =1 $ generally yields promising results, while the best setting varies across datasets, which may rely on the data scale and the concrete scenario.

5.3.2 Batch Size. Considering the context-context contrastive learning task in ContraRec, the number of negative samples in $ \mathcal {L}_{CCC} $ is directly relevant to the training batch size due to in-batch comparison. According to [38], the negative InfoNCE loss will be a tighter lower bound of mutual information between similar instances when the number of negative samples increases. Besides, a larger batch size will make it more possible that different sequences in the batch share the same target item. Therefore, large batch sizes are expected to lead to better results. Figure 7 shows the performance when different batch sizes are adopted. It can be seen that ContraRec indeed benefits from larger batch sizes. The best result is generally achieved with the batch size of 4,096,⁹ while ContraRec is still much better than the base sequential recommendation model with a small batch size of 256.

5.3.3 Temperature. Another important hyper-parameter is the temperature $ \tau _2 $ in Equation (8), which controls the smoothness of the softmax distribution. Figure 8 shows the performance of ContraRec when the temperature ranges from 0.1 to 0.6. We find most settings of $ \tau _2 $ lead to better results than the base sequential encoder, and a small temperature around 0.2 generally yields encouraging results, which is also stable across datasets. When the temperature is small, the model may benefit from focusing more on similar samples while neglecting the ones too far away. In the meantime, too small or especially too large temperatures will also hurt the performance.