| santhosh | |
| Mar 9 2016, 5:26 AM |
| F3752067: pasted_file | |
| Mar 28 2016, 3:18 AM |
After the deployment of AbuseFilter validations, @ssastry informed about a spike in the requests to parsoid to serialize the fragments. This is triggered by the validations in autosave events. The number of autosaves that happen for a long translations can be so high and we may not be efficient in doing this validation every time.
We need to think about some smart ways to reduce frequency of validations.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Reduce the frequency of AbuseFilter Validations | mediawiki/extensions/ContentTranslation | master | +38 -3 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Amire80 | T95655 CX Phase 2 deployment issues (tracking) | |||
| Open | None | T96003 Publishing and saving errors (tracking) | |||
| Open | None | T115375 ContentTranslation publishing failures because of content filtering (tracking) | |||
| Open | None | T114621 Get ContentTranslation to anticipate and handle AbuseFilter warnings and blocks | |||
| Resolved | santhosh | T129314 Tweak the frequency of AbuseFilter validations |
I have the following rough idea:
A potential quick win: do not save unchanged sections right after article is loaded.
The idea of deleting the record from drafts table on first save to cx_corpora table depends on this whole sections save. :/
Can we add a feature flag that indicates when it is coming from drafts or corpora table?
To address the main issue - that is frequency of validations, The above approach I outlined will solve it. The validate flag in translation units becomes true in its 10th edit.
Change 277480 had a related patch set uploaded (by Santhosh):
Reduce the frequency of AbuseFilter Validations
https://gerrit.wikimedia.org/r/277480
Change 277480 merged by jenkins-bot:
Reduce the frequency of AbuseFilter Validations
https://gerrit.wikimedia.org/r/277480
html to wikitext API request rate from Parsoid performance dashboard:
The fix was deployed to production on March 24 and above graph shows immediate decrease in the API request rate