Q1: Employing personal loss distributions doesn’t seem to represent a significant leap forward.

A1: Transitioning from global loss distributions to personal loss distributions is highly meaningful.

• Most existing denoising methods are based on global loss distributions, which present numerous issues, such as significant overlap, thereby limiting denoising performance.
• This problem is particularly severe in BPR-based recommender systems, causing many existing methods to fail (as mentioned in lines 108-127 of the paper).

For example, in Tables 4 and 6, T-CE is effective at denoising in BCE-based recommender systems but significantly hampers recommendation performance in BPR-based systems. Our method is not constrained by the loss function, and we believe this will contribute significantly to the field.

 

---

Q2: What is the R-CE and T-CE's overlap ratio in the training?

A2: We provide the overlap ratios during the training process in the table below.

• We found that R-CE and T-CE even lead to increased overlap ratios in global loss distribution on the MIND dataset within LightGCN (consistent with Table 1 in the paper).
  • This is because the initial overlap causes misclassifications in R-CE and T-CE.
  • Leading to some normal interactions not being involved in training while some noise interactions are, thereby enlarging the overlap region.
• PLD, through personal loss distribution, effectively distinguishes between normal and noisy interactions.
  • Thus, PLD decreases overlap ratios, reflecting better denoising performance.

Epoch - Global Loss Distribution	20	40	60	80	100
Standard Training	18.40	21.32	23.19	25.55	24.32
R-CE	18.79	24.85	27.88	30.62	33.43
T-CE	20.31	26.02	30.18	31.92	35.61
PLD	14.11	15.06	15.33	15.96	16.61

Additionally, we provide the overlap ratios of PLD in the personal loss distribution. We observe that the overlap ratio in the personal loss distribution is significantly lower than that in the global loss distribution, further confirming that considering personal loss distributions is highly meaningful.

Epoch - Personal Loss Distribution	20	40	60	80	100
Standard Training	5.13	5.40	5.29	6.38	6.03
PLD	5.00	5.39	5.32	5.99	6.11

 

---

Q3: While the paper emphasizes personalized denoising, it fails to account for variations in noise levels across different users in its experimental design. All experiments assume a uniform noise ratio for all users, which contradicts the personalized premise of PLD.

A3: We would like to emphasize that:

• "Personalized denoising" refers to our use of each user's personal loss distribution to perform distinct denoising processes for each user, rather than handling personalized noise ratios. This does not conflict with the premise of PLD.
• Additionally, our current experimental setup follows the common settings of previous denoising work [1][2], where noise is proportionally added.
• Therefore, we follow the established settings of previous methods to ensure a fair comparison.

Furthermore, we would like to clarify that:

• In scenarios where different users have varying noise ratios, PLD has a greater advantage over existing methods.
• This is because PLD does not compromise the performance of users with lower noise ratios while still providing sufficient denoising effects (as described in lines 445-449).

To address your concerns, we provide experimental results in scenarios where different users have varying noise ratios (randomly introducing a certain number of noises):

Gowalla	Recall@20
MF	0.1107
+ R-CE	0.1182
+ T-Ce	0.1024
+ DeCA	0.1125
+ BOD	0.1159
+ DCF	0.1110
+ PLD	0.1367
Gain	15.65%

We can observe that, under such conditions, PLD achieves a greater improvement compared to proportionally adding noise.

 

 

• [1] Learning to Denoise Unreliable Interactions for Graph Collaborative Filtering. SIGIR 2022.
• [2] Efficient Bi-Level Optimization for Recommendation Denoising. SIGKDD 2023.

Q4: The performance of baseline methods like DCF is very close to PLD at various noise levels on LightGCN.

A4: Our method, PLD, significantly outperforms the baseline methods.

• To address your concerns, we provide the p-values from T-tests (comparing with the performance of the runner-up) below, demonstrating that PLD significantly outperforms the baseline methods. When the p-value is less than 0.1, PLD has a SIGNIFICANT IMPROVEMENT!

Noise Ratio	0.0	0.1	0.2	0.3	0.4
Gowalla	<1e-7	<1e-7	<1e-7	<1e-5	0.012
Yelp	<1e-6	<1e-5	<1e-5	<1e-5	<1e-5
MIND	<1e-7	<1e-5	<1e-4	0.005	0.09

We can see that all the above results passed the t-test, indicating that PLD significantly outperforms existing methods.

 

---

Q5: The method remains reliant on backbone models like MF and LightGCN.

A5: Our method does not depend on specific backbone models. PLD merely needs to adjust the sampling probability of positive samples during the model training process, exhibiting strong generalization and scalability.

• We selected MF and LightGCN merely because these two methods represent two categories of classical models.
• To address your concern, we have additionally provided experimental results with NeuMF.

Gowalla	Recall@20
NeuMF	0.1203
+ R-CE	0.1295
+ T-Ce	0.0979
+ DeCA	0.1251
+ BOD	0.1267
+ DCF	0.1294
+ PLD	0.1345
Gain	3.94%

We can see that PLD also achieves improvements on different types of backbone models.

 

---

Q6: PLD blurs the distinction between PLD and existing methods like DCF and self-supervised learning techniques.

A6: PLD is distinctly different from existing methods.

• PLD reduces the probability of noise interactions being optimized through resampling.
• Whereas DCF denoises by adjusting weights.
• And self-supervised learning techniques denoise by optimizing different data augmentation views.

We would like to state that PLD is not an exclusive method: it only adjusts the sampling probability of benign samples and is not limited to specific architectures or scenarios.

Therefore, PLD can be combined with existing self-supervised models to achieve better results, as shown in the table below.

Gowalla	Recall@20	Recall@50	NDCG@20	NDCG@50
DCCF (SIGIR23)	0.1649	0.2217	0.1118	0.1329
+ PLD	0.1710	0.2392	0.1151	0.1428
Gain	3.70%	7.90%	2.95%	7.45%

 

---

Q7: Unjustified Addition of Temperature Coefficient $\tau$.

A7: We did not arbitrarily introduce the temperature coefficient $\tau$. We would like to emphasize that:

• We introduced the temperature coefficient $\tau$ through theoretical analysis (lines 502-522 of the paper), specifically:
  • After introducing $\tau$, we can increase $\mathbb{E}[{\mathrm{\Lambda }}_{\text{normal}}-{\mathrm{\Lambda }}_{\text{noise}}]$ to achieve better performance.
• Additionally, the introduction of $\tau$ does not affect the conclusion of Theorem 1; it only requires substituting ${\mu }_1^{\prime }={\mu }_1/\tau$, ${\mu }_2^{\prime }={\mu }_2/\tau$, and ${\sigma }^{\prime 2}={\sigma }^2/{\tau }^2$ into Equation 2.

 

---

Q8: Baseline Comparisons Lack Cutting-Edge Methods.

A8: The methods we compared against (e.g., R-CE, T-CE, DeCA, BOD - SIGKDD23, DCF - SIGKDD24) include the latest state-of-the-art methods up to 2024.

• These comparative methods are directly relevant to PLD.
• Regarding the "self-supervised learning techniques and graph-based denoising" you mentioned, such methods do not conflict with the aforementioned denoising methods and can serve as backbone models in combination with these methods to achieve better results, as demonstrated in the table in A6 with DCCF+PLD.

Q9: Noise Distribution Variation Across Users.

A9: We would like to emphasize that our experimental setup follows the common settings of existing work [1][2]. In response to your questions, we provide the following answers:

 

• Q9-1: How would the performance of PLD change if the noise level varied across users rather than being uniformly distributed?

  A9-1: When noise levels vary across users, PLD still achieves consistently good performance and even shows more significant improvements, as demonstrated in the table in A3.

  • This is because PLD does not compromise the performance of users with lower noise ratios while still providing sufficient denoising effects (as described in lines 445-449 of the paper).
  • Therefore, following the established settings of uniform noise ratio in previous works can ensure a fair comparison.

 

• Q9-2: Does the stability of PLD's performance gap with LightGCN across noise levels indicate that a uniform noise assumption is overly favorable to PLD?

  A9-2: Not at all. In fact, scenarios with varying noise levels across users are more favorable to PLD (as described in lines 445-449 of the paper).

  • As shown in the table in A3, PLD performs well when noise levels vary among users. The results indicate that this setup is more advantageous for PLD rather than the uniform noise assumption.

 

• Q9-3: Would introducing personalized noise levels better reflect the real-world applicability of PLD and align more closely with the personal denoising concept?

  A9-3:

  Firstly, we agree that introducing different noise levels for different users may be more consistent with real-world scenarios.

  • However, we set the same noise level in our experiments to provide a fairer comparison, even though this scenario is more favorable to previous methods.
  • We will add the results for different noise levels to the revised paper to provide a more comprehensive comparison.

  Secondly, we would like to reiterate that "personalized denoising" refers to our use of each user's personal loss distribution to perform distinct denoising processes for each user, rather than handling personalized noise ratios.

 

 

---

Q10: Is PLD fundamentally different from data augmentation strategies like those in DCF or self-supervised learning methods?

A10: PLD is distinctly different from existing methods.

• PLD reduces the probability of noise interactions being optimized through resampling.
• Whereas DCF denoises by adjusting weights.
• Self-supervised learning techniques denoise by optimizing different data augmentation views.
• Additionally, PLD can be integrated with existing self-supervised learning models to provide better performance, as demonstrated in A6.

 

• Q10-1: Given that PLD relies on backbone models (MF, LightGCN), how does it address the limitations inherent to those models?

  A10-1: PLD does not depend on specific backbone models.

  • It is a resampling approach that improves the training process of the model without relying on specific model architectures.
  • PLD can be added to traditional recommendation models (MF, LightGCN, NeuMF as shown in A5) and to self-supervised models (DCCF as shown in A6).

 

• Q10-2: Does the convergence of PLD's performance with DCF at higher noise levels suggest that the resampling strategy adds minimal value beyond existing approaches?

  A10-2:

  Firstly, we would like to clarify that we adopt 40% and 50% noise levels, which are already far beyond realistic noise levels, solely to verify robustness in extreme scenarios. Such high noise ratios are generally unlikely to occur in real-world applications.

  • We set such high noise ratios solely to demonstrate the robustness of different methods against noise.
  • When the noise ratio is excessively high, it severely degrades backbone model performance, resulting in smaller differences between methods.

  Secondly, in addition to PLD's superior performance improvements, we also demonstrate in Figure 9 and Appendix A.2 that PLD has lower time and space complexity. Overall, PLD provides substantial value.

 

 

• [1] Learning to Denoise Unreliable Interactions for Graph Collaborative Filtering. SIGIR 2022.
• [2] Efficient Bi-Level Optimization for Recommendation Denoising. SIGKDD 2023.

Q11: Temperature Coefficient $\tau$:

• Q11-1: How does the introduction of $\tau$ alter the mathematical formulation of $\xi$ in the resampling probability?

  A11-1: After introducing $\tau$, we can express $\xi$ as $\xi =\exp (({\mu }_1-{\mu }_2)/\tau )$, as shown in line 529 of the paper.

  By decreasing $\tau$, we can reduce $\xi$, thereby actively increasing $\frac{n\alpha -m\beta }{(m+n)\eta }$ to enhance the sampling probability of normal interactions, as shown in lines 530-531 of the paper.

 

• Q11-2: Can the authors provide a theoretical justification for how $\tau$ interacts with other hyperparameters to influence denoising performance?

  A11-2: Firstly, The introduction of $\tau$ does not affect the conclusions of Theorem 1.

  • We only need to substitute ${\mu }_1^{\prime }={\mu }_1/\tau$, ${\mu }_2^{\prime }={\mu }_2/\tau$, and ${\sigma }^{\prime 2}={\sigma }^2/{\tau }^2$ into $\alpha$, $\beta$, and $\gamma$ in Theorem 1 without modifying other expressions.
  • Additionally, after introducing $\tau$, we still have $\mathrm{\Gamma }>\frac{\chi }{C^2}\frac{k^2}{(k-1{)}^2}$, so it does not affect the subsequent analysis with $k$.

  Secondly, in conjunction with the response in A11-1, by decreasing $\tau$, we can actively increase $\frac{n\alpha -m\beta }{(m+n)\eta }$ to enhance the sampling probability of normal interactions, while reducing the sampling probability of noisy interactions, as shown in lines 529-531 of the paper.

 

 

---

Q12: Baseline Methods:

A12: The methods we compared against (e.g., R-CE, T-CE, DeCA, BOD - SIGKDD23, DCF - SIGKDD24) include the latest state-of-the-art methods up to 2024.

• These comparative methods are directly relevant to PLD.
• Regarding the "self-supervised learning techniques and graph-based denoising" you mentioned, such methods do not conflict with the aforementioned denoising methods and can be combined to achieve better results, as shown in A8.

 

 

---

Thank you for your detailed comments and suggestions, as well as your appreciation of the importance of this work. We hope that our clarifications address your concerns. Based on our responses, we kindly hope that you can reconsider the overall assessment of this paper. We believe that our work offers valuable insights and advances the state-of-the-art in robust recommender systems, making a meaningful contribution to the research community.

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have addressed your concerns. We look forward to your feedback.

Thank you once again for your time and effort.

Sincerely,
The Authors

We sincerely thank you for your time and effort in reviewing our paper and supporting our work. With the discussion period deadline (Dec. 10th) approaching, we are following up on our earlier response.

We understand your schedule may be busy, and we appreciate the time you've already invested in reviewing our paper. We have carefully considered your comments and provided detailed clarifications and additional support for each concern in our response.

We would be grateful for your feedback on whether our response has addressed your concerns. If your concerns have been addressed, we kindly request that you consider re-evaluating our paper and increasing our scores. Your continued support is immensely valuable, and we truly appreciate your feedback.

Once again, thank you for your insights and ongoing support.

Sincerely,
The Authors

Thanks for the rebuttal, which addresses many technical concerns. However, I still concern about the novelty of this paper, since there are already many denoising papers based on the loss scale. I admit that the finding that using the user-based loss is important and theoretical guarantees exist, Thus, I will increase the rating of Technical Quality to 4, and maintain the Novelty rating as 3.

Thank you for your response and for acknowledging the theoretical contributions of our work.

However, we would like to emphasize an important point. While some prior works have employed loss-based approaches for denoising, these methods suffer from inherent limitations, such as significant overlap issues and challenges in applying them to BPR loss-based recommender systems.

In contrast, we propose a novel perspective by leveraging personal loss distribution, which we have theoretically demonstrated to be superior. Moreover, our experimental results clearly show that our method consistently outperforms traditional approaches across various scenarios.

Thus, it is unreasonable to dismiss the novelty of subsequent loss-based denoising methods solely on the grounds that "prior works have explored loss-based approaches."

It is also worth noting that Michael J. Black emphasized in Novelty in Science: “Taking an existing network and replacing one thing is better science than concocting a whole new network just to make it look more complex” [https://perceiving-systems.blog/en/news/novelty-in-science]. This principle has also been incorporated into the review guidelines of many conferences, such as [https://logconference.org/reviews/#best-practices]. We believe that our simple yet effective approach embodies this principle and offers genuine novelty.

We belive that our work provides a innovative perspective and a novel approach to denoising, as also recognized by other reviewers (reviewers #2, #4, #5). Furthermore, in your previous review, you also acknowledged the novelty of our method and recognized the innovation of applying personal loss distribution.

In light of this, we sincerely hope you will reconsider your evaluation and assign a higher score to our work.

Sincerely,
The Authors

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have addressed your concerns. We look forward to your feedback.

Thank you once again for your time and effort.

Sincerely,
The Authors

We sincerely thank you for your time and effort in reviewing our paper and supporting our work. With the discussion period deadline (Dec. 10th) approaching, we are following up on our earlier response.

We understand your schedule may be busy, and we appreciate the time you've already invested in reviewing our paper. We have carefully considered your comments and provided detailed clarifications and additional support for each concern in our response.

We would be grateful for your feedback on whether our response has addressed your concerns. If your concerns have been addressed, we kindly request that you consider re-evaluating our paper and increasing our scores. Your continued support is immensely valuable, and we truly appreciate your feedback.

Once again, thank you for your insights and ongoing support.

Sincerely,
The Authors

Thanks to the author for the response, which addresses most of my concerns. Since my scores are relatively high, I tend to maintain the score. I’ll discuss further with the other reviewers and the AC during the discussion stage.

Thank you for reviewing our response. We are pleased to hear that our response has addressed your concerns. Please don’t hesitate to reach out if you have any further questions.

Sincerely,
Authors

Q1: The paper lacks an in-depth explanation of why the person-level loss has less overlap than the overall loss.

A1: As mentioned in our previous reply, we have conducted a detailed analysis in lines 367-378 of the manuscript, specifically:

• The data contains some hard-to-train users (high-loss users) and some easy-to-train users (low-loss users), as shown in Figure 3 of the paper.
• This causes the loss of normal interactions for high-loss users to potentially exceed the loss of noisy interactions for low-loss users.
• Consequently, in the Global Loss Distribution, a higher number of normal interactions have higher losses than noisy interactions.
• In contrast, personal loss distributions mitigate the issue of varying losses among users, thereby reducing the overlap.

To further substantiate this conclusion, we introduced varying levels of noise for different users, which amplifies the differences in losses among users. Under these conditions, in the Gowalla dataset:

• The proportion of Normal Interaction within Overlap in the Global Loss Distribution increased from 15.33% to 26.07%.
• The proportion of Normal Interaction within Overlap in the Personal Loss Distribution remained nearly unchanged (4.11% and 4.3%).

These results confirm the stronger advantage of Personal Loss Distributions.

Additionally, we present the corresponding performance in the table below: We observed that under these conditions, PLD achieved greater improvements, further demonstrating the advantages of personalized loss distributions.

Gowalla	Recall@20
MF	0.1107
+ R-CE	0.1182
+ T-CE	0.1024
+ DeCA	0.1125
+ BOD	0.1159
+ DCF	0.1110
+ PLD	0.1367
Gain	15.65%

 

---

Q2: Theorem 1 assumes that the loss follows a Gaussian distribution, but there is no further explanation. Is it possible to remove the Gaussian assumption in Theorem 1?

A2: We would like to clarify that we verified that personal loss distribution can be approximated by a Gaussian distribution by the Shapiro-Wilk test; furthermore, we can remove the Gaussian assumption in Theorem 1.

Firstly, the personal loss distribution can be approximated by a Gaussian distribution, as illustrated in Figure 3 of the main text.

• To address your concern, we conducted the Shapiro-Wilk test on the normal interactions of the personal loss distributions (a higher p-value indicates better conformity to a Gaussian distribution).
  • At a significance level of 0.05, 65.89% of users' data accepted the normal distribution assumption.
  • At a more stringent significance level of 0.005, 78.98% of users' data did so.

Secondly, the assumption of a Gaussian distribution in Theorem 1 primarily facilitates the derivation of specific probability expressions and does not affect the conclusions.

• Specifically, as detailed in the proof in the appendix (lines 1198-1199), the derived expressions do not rely on any particular distributional assumptions:
  • $\mathbb{E}[{\mathrm{\Lambda }}_{\text{normal}}]=\mathbb{E}[S_x]\cdot \mathbb{E}\left[\frac{1}{S_x+S_y}\right]-\frac{kn}{n+m}\cdot \frac{\mathrm{Var}[\exp (-x_i)]}{(k-1{)}^2{C}^2},$
  • where $\mathbb{E}\left[\frac{1}{S_x+S_y}\right]\approx \frac{1}{\mathbb{E}[S_x]+\mathbb{E}[S_y]}(1+\frac{\mathrm{Var}[S_x]+\mathrm{Var}[S_y]}{{(\mathbb{E}[S_x]+\mathbb{E}[S_y])}^2}).$
  • Above expressions do not rely on any particular distributional assumptions.
• Similarly, in Equation (2), different distributions can be accommodated by substituting the corresponding means and variances into the parameters $\alpha$, $\beta$, and $\gamma$, without altering the conclusion of Theorem 1.

Due to space constraints, we will discuss this point in more detail in the revised manuscript.

Q3: Is it possible to provide a theorem for proving the person-level loss distribution has less overlap than the overall loss?

A3: Certainly. To maintain brevity during the rebuttal phase, we provide a concise statement of the theorem below:

Assumptions:

1. The means of all users' losses follow a distribution with mean ${\mu }_o$ and variance ${\sigma }^2$.
2. For a given user, the mean loss is ${\mu }_u$. We assume:
   • The loss of normal interactions follows a distribution with mean ${\mu }_u^{+}={\mu }_u+{ϵ}^{+}$ and variance ${\sigma }_u^{+}$.
   • The loss of noisy interactions follows a distribution with mean ${\mu }_u^{-}={\mu }_u-{ϵ}^{-}$ and variance ${\sigma }_u^{-}$.
   • Here, ${ϵ}^{+},{ϵ}^{-}\sim \mathcal{N}(0,1)$ serves as an adjustment term.

Conclusion: When ${\sigma }^2>{\sigma }_u^{+},{\sigma }_u^{-}$ for $u\in \mathcal{U}$ (which is obviously satisfied), by calculating the sum of the expectation of overlap for each user's personalized loss distribution and comparing it to the expectation of the overlap in the global loss distribution, we demonstrate that the overlap in personalized loss distributions is smaller than that in the global loss distribution.

This theorem underscores that personalized loss distributions effectively reduce overlap, thereby enhancing the discriminative power between normal and noisy interactions.

 

---

Thank you for your detailed comments and useful suggestions, as well as your appreciation of the importance of this work. We hope that our clarifications address your concerns. We look forward to your reconsideration and reevaluation of the overall assessment of our work.

Thanks for your quick response.

To Gaussian Assumption

• I believe you may have misunderstood the results of the Shapiro-Wilk test. As far as I know, a high p-value only indicates that we cannot reject the Gaussian hypothesis (e.g., the sample count may be too small to draw a conclusion, which is likely the case). It does not mean that we can accept the Gaussian hypothesis. Your analysis demonstrates that even with a very small sample count (shown in your Figure 3), as high as 34.11% of user losses can reject the Gaussian hypothesis, which actually disproves your assumption.

• However, I appreciate your claim that Theorem 1 can apply to various distributions. I suggest removing the assumption that the loss follows a Gaussian distribution from the paper and revising Theorem 1 accordingly.

To Overlap Assumption

• I appreciate your intuitive explanation of the overlap phenomenon. I believe emphasizing these key explanations (e.g., the presence of high-loss users and low-loss users) in the Introduction and Motivation sections could significantly enhance the persuasiveness of the paper.

• Although I appreciate the new theorem you introduced, I have concerns regarding your assumption about the adjustment term. You assume that the variance of epsilon is 1, which is independent of the scale of ${\mu }_u$ and counterintuitive. For example, in an extreme case, if ${\mu }_u$ ranges on the order of 10^2, the impact of adding epsilon would be negligible; however, if ${\mu }_u$ is around 0.01, adding epsilon would significantly affect the results. Moreover, why do these mathematical assumptions fail to reflect the statement "presence of high-loss users and low-loss users"? This seems to contradict your intuitive explanation.

Thank you for your insightful feedback. Indeed, a subset of users reject the assumption of a Gaussian distribution. Due to the extreme sparsity of interactions for each user, we are unable to perform further analysis under the Gaussian distribution framework. Consequently, we will remove the assumption that the loss follows a Gaussian distribution from the paper and revise Theorem 1 accordingly. We greatly appreciate your suggestion, as it enhances the theoretical rigor of our work.

 

Regarding the new theorem provided in A3, we assumed $ϵ\sim \mathcal{N}(0,1)$ for the sake of simplifying the derivation. We apologize for not explicitly stating the condition ${\mu }_o\gg 1$ that underpins our analysis:

• Under the condition ${\mu }_o\gg 1$, the new theorem holds true.
• Should this condition need to be relaxed, we could consider:
  • Setting $ϵ\sim \mathcal{N}(0,1\times {10}^{-4}{\mu }_o^2)$.
  • Or, defining ${\mu }_u^{+}={\mu }_o(1+{ϵ}^{+})$ and similarly adjusting ${\mu }_u^{-}$.

 

Furthermore, the distinction between "high-loss users" and "low-loss users" is valid. For example:

• Low-loss users: A subset of users have a loss mean ${\mu }_{u1}<{\mu }_o-2\sigma$ (with 2 serving as an illustrative value).
• High-loss users: Another subset of users have a loss mean ${\mu }_{u2}>{\mu }_o+2\sigma$.

In such scenarios, the loss from noisy interactions of low-loss users exceeds that of normal interactions from high-loss users. Specifically, when $\sigma >{\sigma }_u^{+}$ and $\sigma >{\sigma }_u^{-}$ (the conditions we provide in A3), this phenomenon is further ensured:

• That is, for low-loss and high-loss users, the expectation of loss from noisy interactions of low-loss users is guaranteed to be higher than the expectation of loss from normal interactions of high-loss users.

 

Once again, thank you for your valuable feedback. We hope that the clarifications provided address your concerns. We kindly request you to reconsider your overall assessment of our paper. We believe our research offers significant insights and advances the state-of-the-art in robust recommender systems, thereby making a meaningful contribution to the research community.

Thank you for your clarification. However, assuming ${\mu }_o\gg 1$ is undoubtedly not realistic, since most of the loss values in your Figure 1 are less than 1. Also, Setting $ϵ\sim \mathcal{N}(0,{10}^{-4}{\mu }_o^2)$ or defining ${\mu }_u={\mu }_o(1+{ϵ}^{+})$ also seems to too strong and unnatured.

What about using $ϵ\sim \mathcal{N}(0,k)$ where k is some constant? Under this condition, can you prove your theorem when $k$ is sufficiently large?

Thank you for your response and engaging discussion. Firstly, I would like to correct the relaxed condition I provided in my previous reply: it should be ${\mu }_u^{+}={\mu }_u(1+{ϵ}^{+})$ instead of ${\mu }_o(1+{ϵ}^{+})$. This condition implies that ${\mu }_u^{+}$ and ${\mu }_u^{-}$ are obtained by fine-tuning ${\mu }_u$.

 

Next, we will explore the implications of the theorem under the assumption that $ϵ\sim \mathcal{N}(0,k)$. We aim to address this issue from two perspectives:

1. The conclusion of the theorem when $k$ is sufficiently large.
2. The magnitude of $k$ in practical scenarios.

 

1. When $k$ is Sufficiently Large

Firstly, when $k$ is excessively large, i.e., $k\gg {\mu }_o^2$, we are unable to rigorously prove that the aforementioned theorem holds. Through approximation, we can assert that the theorem holds with a certain probability; however, the specific probability would require further assumptions about the distribution outlined in A3.

 

Alternatively, we could introduce a new condition, ${\mu }_u^{+}<{\mu }_u^{-}$ (a condition derived from our observations). Specifically:

• ${\mu }_u^{+}={\mu }_u-|{ϵ}^{+}|$, ${\mu }_u^{-}={\mu }_u+|{ϵ}^{+}|$

Under this condition:

• Regardless of the value of $k$, we can ensure that the global overlap is greater than or equal to the personal overlap.
• Specifically, when $k$ is less than $\sqrt{\pi }\mathbb{E}[{\mu }_u^2]$, the equality can be removed.
  • Calculating $\mathbb{E}[{\mu }_u^2]$ requires a combination of ${\mu }_o$ and ${\sigma }^2$, which requires further assumptions about the distribution.

 

Of course, this assumption may be somewhat strong, so we can consider relaxing this condition:

• If a certain percentage (namely $\beta$) of users satisfy ${\mu }_u^{+}<{\mu }_u^{-}$, we can still achieve that the global overlap is greater than or equal to the personal overlap.
• Specifically, we could introduce a parameter $\gamma$ when setting ${\mu }_u^{+}$ and ${\mu }_u^{-}$, thereby adjusting the mean of the distribution sampled by $ϵ$ by adding or subtracting $\gamma$ to control $\beta$:
  • However, the specific expression for this percentage would require further assumptions about the distribution in A3.
  • Generally, when $\beta$ is larger than a value related to ${\mu }_o,\sigma ,k$, it is guaranteed that the above conclusion is valid.
  • According to Figure 4, more than 95 percent of users meet ${\mu }_u^{+}<{\mu }_u^{-}$. Thus, the above conditions are very easy to meet.

 

2. Magnitude of $k$ in Practical Scenarios

A larger $k$ implies that the difference between users' normal interaction loss and noisy interaction loss may be greater. Based on our experience and the illustrations provided in the paper:

• In Figure 1 (BPR loss), we observe that the loss ranges between 0 and 3.0, with the mean approximately 0.1618.
• In Figure 4, the absolute difference (calculation by quartiles as explained in lines 383-386 in the paper) between normal interaction loss and noisy interaction loss lies between 0 and 1.0.
  • When we calculate by using the mean values, we get that these differences lies between 0 and 0.64.
  • We can assertively conclude that:
    • When $3\sqrt{2k}\approx 0.64$, i.e., $k\approx 0.0228$, there is a 99.7% probability that the difference lies between 0 and 0.64.
    • (This approximation assumes a Gaussian distribution for the Personal Distribution solely to provide a numerical example to show that k is not too large).
• Therefore, the scenario where $k\gg {\mu }_o^2$ is highly unlikely to occur, and the theorem mentioned earlier can effectively disregard the impact of such extreme cases.

 

Once again, thank you for your valuable feedback. If you have any further questions, please do not hesitate to let me know, and I would be happy to engage in a deeper discussion with you. Additionally, we hope you will consider increasing the score of our work, as our methods have been empirically and theoretically validated and we believe they make a meaningful contribution to the research field.

Thanks for the response. These theoretical analyses are interesting and may further strengthen this paper's motivation. I think they may also inspire other scenarios with similar heterogeneous loss. Although other reviewers' opinions may be valid, I have decided to raise the scores (novelty, technical quality) from (4, 4) to (5, 6).

Thank you for taking the time to review our response and for considering an increased score for our paper. We are grateful for the insightful discussion and your valuable comments on improving our work.

Sincerely,
Authors

Q1: The authors fail to provide sufficient empirical evidence to support the assumptions proposed in this work.

A1: Our work indeed provides empirical evidence to support these assumptions. We present user cases (Figure 3) and statistical evidence (Figures 1 and 2, Figures 4, Tables 1 and 2).

• See Figures 1 and 2, Table 1: Our work identifies that in the global loss distribution:
  • The loss of noisy interactions is higher than that of normal interactions.
• See Figures 3 and 4, Table 2: We present evidence demonstrating that in the personal loss distribution:
  • The loss of noisy interactions is higher than that of normal interactions.
  • Theorem 1 provides our method with superior denoising guarantees.

Additionally, we have already provided statistical results on three different scenario datasets as shown in previous rebuttal (Gowalla Check-in dataset, Yelp Review dataset, MIND News dataset), verifying that in both global and personal loss distributions, the loss of noisy interactions is higher than that of normal interactions. We believe this further alleviates your concerns.

Overall, we have provided denoising method PLD assurances from both empirical and theoretical perspectives. As Reviewer #2 acknowledged, our data analysis offers sufficient evidence to highlight the shortcomings of existing methods and support our assumptions.

 

---

Q2: Theoretical analysis heavily relies on intuitive reasoning and unproven assumptions.

A2: We would like to clarify that:

Regarding Assumptions, we used two assumptions in this theorem:

1. Assumption that the loss of noisy interactions is higher than that of normal interactions: We have clarified this repeatedly in previous replies, citing evidence from the paper and providing additional data to support this. We have validated that in both global and personal loss distributions, the loss of noisy interactions is higher than that of normal interactions.

2. Assumption that the personal loss distribution follows a Gaussian distribution:

   1. As illustrated in Figure 3 of the main text, the personal loss distribution approximates a Gaussian distribution. And we conducted the Shapiro-Wilk test on the normal interactions within the personal loss distributions (a higher p-value indicates better conformity to a Gaussian distribution).
      1. At a significance level of 0.05, 65.89% of users' data accepted the normal distribution assumption.
      2. At a more stringent significance level of 0.005, 78.98% of users' data did so.
      3. This proves that the personal loss distribution follows a Gaussian distribution.
   2. Additionally, this assumption primarily facilitates the derivation of specific probability expressions and does not affect the overall conclusions.
      1. Specifically, you can refer to the proof in the appendix (lines 1198-1199), where the expressions do not rely on any particular distributional assumptions:
         • $\mathbb{E}[{\mathrm{\Lambda }}_{\text{normal}}]=\mathbb{E}[S_x]\cdot \mathbb{E}\left[\frac{1}{S_x+S_y}\right]-\frac{kn}{n+m}\cdot \frac{\mathrm{Var}[\exp (-x_i)]}{(k-1{)}^2{C}^2},$
         • where $\mathbb{E}\left[\frac{1}{S_x+S_y}\right]\approx \frac{1}{\mathbb{E}[S_x]+\mathbb{E}[S_y]}(1+\frac{\mathrm{Var}[S_x]+\mathrm{Var}[S_y]}{{(\mathbb{E}[S_x]+\mathbb{E}[S_y])}^2}).$
         • Above expressions do not rely on any particular distributional assumptions.
      2. Similarly, in Equation (2), when using different distributions, only the corresponding means and variances need to be substituted into $\alpha$, $\beta$, and $\gamma$, without altering the conclusion of Theorem 1.

We would like to emphasize that our proofs have undergone rigorous mathematical derivation (see Appendix A.1). The rigor of our theoretical analysis has been commended by all other reviewers. We believe that certain misunderstandings may have led to this misjudgment.

Q3: The simplistic reweighting design and the lack of modeling for complex implicit feedback make it difficult to fully trust the method’s effectiveness.

A3: First, we would like to clarify that our resampling method can be integrated with any loss function and do model the complex implicit feedback. This simple but effective design enhances the method's generalizability and scalability.

To further illustrate this point, we provide experimental results using NeuMF (NeuMF models feedback through a neural network), which models more complex feedback compared to dot-product models:

Gowalla	Recall@20
NeuMF	0.1203
+ R-CE	0.1295
+ T-Ce	0.0979
+ DeCA	0.1251
+ BOD	0.1267
+ DCF	0.1294
+ PLD	0.1345
Gain	3.94%

Additionally, we provide results for PLD integrated with a self-supervised backbone model, DCCF (SIGIR 2023). Due to the complexity of these backbone models, not all baselines could be incorporated. Therefore, we have prioritized providing results for PLD as shown below:

Gowalla	Recall@20	Recall@50	NDCG@20	NDCG@50
DCCF (SIGIR23)	0.1649	0.2217	0.1118	0.1329
+ PLD	0.1710	0.2392	0.1151	0.1428
Gain	3.70%	7.90%	2.95%	7.45%

Our method, PLD, achieved state-of-the-art performance across these models, demonstrating that PLD do model the complex implicit feedback inherent in backbone models.

Additionally, the theoretical guarantees we provide support the effectiveness of our approach.

 

---

Q4: The paper is based on strong assumptions but lacks a detailed discussion of the proposed method's limitations and its applicability to specific usage scenarios.

A4: We would like to clarify agian that the assumptions our work relies on have been well-validated, as evidenced by

• User cases (Figure 3)
• Statistical evidence (Figures 1-2, 4, Tables 1-2).
• Additionally, in our previous replies, we have provided statistical data across different datasets.

These pieces of evidence confirm that in the personal loss distribution, the loss of noisy interactions is higher than that of normal interactions.

Our method merely participates in the model training process, adjusting the sampling probability of positive samples. Compared to other methods, our approach offers greater flexibility and adaptability to different backbone models. In A3, we also provided results on backbone models of varying complexity.

In real-world scenarios, behaviors such as misclicks by some users can introduce significant noise into the data. In such cases, by incorporating PLD and adjusting the sampling probability of positive samples during model training, we can achieve improved performance. This has been validated in Section 5.2 of our paper.

We'll further include these discussion in the revised paper.

 

---

Thank you for your detailed comments and useful suggestions, as well as your appreciation of the importance of this work. We believe that the above responses will solve your misunderstandings regarding the assumptions used in our method. We look forward to your reconsideration and reevaluation of the overall assessment of our work.

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have addressed your concerns. We look forward to your feedback.

Thank you once again for your time and effort.

Sincerely,
The Authors

Thank you for the authors' patient response. I have revised the score.

Thank you for reviewing our response and for your willingness to increase the score of our paper. We are pleased to hear that our response has addressed your concerns. Please don’t hesitate to reach out if you have any further questions.

Sincerely,
The Authors

Thank you for taking the time to review our response and for considering an increased score for our paper. We are grateful for your valuable comments to improve our work.

Sincerely,
Authors

Q1: Potentially inappropriate assumption of homoscedastic Gaussian distribution in Theorem 1.

A1: Thank you for your comment!

Firstly, we would like to clarify that the Gaussian distribution assumption we introduced pertains to the personal loss distribution, whereas Figures 1 and 2 illustrate the global loss distribution. We do not make any assumptions about the global loss distribution.

Secondly, regarding the assumption of a homoscedastic Gaussian distribution in Theorem 1, it is primarily introduced to simplify the derivation and enhance the clarity of the expressions.

• This assumption does not affect the conclusion of the theorem.
• Specifically, you can refer to the proof in the appendix (lines 1198-1199), where the expressions for $\mathbb{E}[\mathrm{\Lambda }]$ do not rely on any specific distributional assumptions:
  • $\mathbb{E}[{\mathrm{\Lambda }}_{\text{normal}}]=\mathbb{E}[S_x]\cdot \mathbb{E}\left[\frac{1}{S_x+S_y}\right]-\frac{kn}{n+m}\cdot \frac{\mathrm{Var}[\exp (-x_i)]}{(k-1{)}^2{C}^2},$
  • where $\mathbb{E}\left[\frac{1}{S_x+S_y}\right]\approx \frac{1}{\mathbb{E}[S_x]+\mathbb{E}[S_y]}(1+\frac{\mathrm{Var}[S_x]+\mathrm{Var}[S_y]}{{(\mathbb{E}[S_x]+\mathbb{E}[S_y])}^2}).$
  • Above expressions do not rely on any particular distributional assumptions.
• Similarly, if different variances (${\sigma }^2$) are to be used in Theorem 1, the corresponding values can be substituted into $\alpha$ and $\beta$, while distinguishing between ${\gamma }_1$ and ${\gamma }_2$.
• After using different variances (${\sigma }^2$), the mathematical form of Equation (2) does not change, and the conclusion remains unaffected.

This substitution does not affect the conclusion of Theorem 1. Due to space limitations, we will discuss this point in more detail in the revised manuscript.

 

---

Q2: Selection of backbone is limited.

A2: Thank you for your suggestion! To address your concern, we have additionally provided experimental results using NeuMF as a backbone model.

Gowalla	Recall@20
NeuMF	0.1203
+ R-CE	0.1295
+ T-Ce	0.0979
+ DeCA	0.1251
+ BOD	0.1267
+ DCF	0.1294
+ PLD	0.1345
Gain	3.94%

Furthermore, we have explored experimental results with more advanced backbone models, DCCF. Due to the complexity of these backbone models, not all baselines could be incorporated. Therefore, we have prioritized providing results for PLD as shown below:

Gowalla	Recall@20	Recall@50	NDCG@20	NDCG@50
DCCF (SIGIR23)	0.1649	0.2217	0.1118	0.1329
+ PLD	0.1710	0.2392	0.1151	0.1428
Gain	3.70%	7.90%	2.95%	7.45%

Our method, PLD, achieved improvements across different backbones, demonstrating its effectiveness and generalizability.

 

---

Q3: Considering that the current placement of the legend is farther away from the corresponding bar chart?

A3: Thank you for your suggestion! We will adjust the position of the legend in the revised version to ensure it is closer to the corresponding bar charts, thereby enhancing readability.

Q4: Why does the difference between BPR loss and BCE loss lead to such a significant discrepancy in the results? It might be useful to explore the results on other datasets under the BCE loss setting.

A4: Thank you for your question! As mentioned in lines 121-127 of our paper, the overlap phenomenon is further exacerbated when using BPR loss, which negatively impacts the effectiveness of methods based on global loss distributions.

This is because:

• BCE Loss: BCE loss is a point-wise loss, allowing us to directly compute the loss of each interaction.
• BPR Loss: BPR loss is a pair-wise loss, where computing the loss of each interaction depends on negative samples. Introducing negative samples increases the variances of the global loss distribution.
• Consequently, the loss of normal interactions for difficult-to-train users (high-loss users) may exceed that of noisy interactions for easy-to-train users (low-loss users), resulting in greater overlap.
• This makes global loss distributions less effective in distinguishing normal interactions from noisy interactions, thereby rendering denoising methods that rely on global loss distributions ineffective.

To address your concern, we provide additional results on the Gowalla dataset using the BCE loss setting. We found that PLD still achieves the best performance on the Gowalla dataset.

Gowalla	Recall@20
MF-BCE	0.1109
+ R-CE	0.1306
+ T-Ce	0.1319
+ DeCA	0.1310
+ BOD	0.1361
+ DCF	0.1351
+ PLD	0.1456
Gain	6.98%

 

---

Thank you for your detailed comments and useful suggestions, as well as your appreciation of the importance of this work. We hope that our clarifications address your concerns. We look forward to your reconsideration and reevaluation of the overall assessment of our work.

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have addressed your concerns. We look forward to your feedback.

Thank you once again for your time and effort.

Sincerely,
The Authors

We sincerely thank you for your time and effort in reviewing our paper and supporting our work. With the discussion period deadline (Dec. 10th) approaching, we are following up on our earlier response.

We understand your schedule may be busy, and we appreciate the time you've already invested in reviewing our paper. We have carefully considered your comments and provided detailed clarifications and additional support for each concern in our response.

We would be grateful for your feedback on whether our response has addressed your concerns. If your concerns have been addressed, we kindly request that you consider re-evaluating our paper and increasing our scores. Your continued support is immensely valuable, and we truly appreciate your feedback.

Once again, thank you for your insights and ongoing support.

Sincerely,
The Authors