research-article

Controllable and Diverse Text Generation in E-commerce

Authors:

Tarek AbdelzaherAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

Pages 2392 - 2401

https://doi.org/10.1145/3442381.3449838

Published: 03 June 2021 Publication History

Abstract

In E-commerce, a key challenge in text generation is to find a good trade-off between word diversity and accuracy (relevance) in order to make generated text appear more natural and human-like. In order to improve the relevance of generated results, conditional text generators were developed that use input keywords or attributes to produce the corresponding text. Prior work, however, do not finely control the diversity of automatically generated sentences. For example, it does not control the order of keywords to put more relevant ones first. Moreover, it does not explicitly control the balance between diversity and accuracy. To remedy these problems, we propose a fine-grained controllable generative model, called Apex, that uses an algorithm borrowed from automatic control (namely, a variant of the proportional, integral, and derivative (PID) controller) to precisely manipulate the diversity/accuracy trade-off of generated text. The algorithm is injected into a Conditional Variational Autoencoder (CVAE), allowing Apex to control both (i) the order of keywords in the generated sentences (conditioned on the input keywords and their order), and (ii) the trade-off between diversity and accuracy. Evaluation results on real world datasets 1 show that the proposed method outperforms existing generative models in terms of diversity and relevance. Moreover, it achieves about 97% accuracy in the control of the order of keywords.

Apex is currently deployed to generate production descriptions and item recommendation reasons in Taobao2, the largest E-commerce platform in China. The A/B production test results show that our method improves click-through rate (CTR) by 13.17% compared to the existing method for production descriptions. For item recommendation reason, it is able to increase CTR by 6.89% and 1.42% compared to user reviews and top-K item recommendation without reviews, respectively.

References

[1]

Karl Johan Åström, Tore Hägglund, and Karl J Astrom. 2006. Advanced PID control. Vol. 461. ISA-The Instrumentation, Systems, and Automation Society Research Triangle.

[2]

Karl Johan Åström, Tore Hägglund, Chang C Hang, and Weng K Ho. 1993. Automatic tuning and adaptation for PID controllers-a survey. Control Engineering Practice 1, 4 (1993), 699–714.

[3]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72.

[4]

Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. 2015. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349(2015).

[5]

Zhangming Chan, Xiuying Chen, Yongliang Wang, Juntao Li, Zhiqiang Zhang, Kun Gai, Dongyan Zhao, and Rui Yan. 2019. Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation. In Proceedings of the 2019 EMNLP. 4960–4969.

[6]

Qibin Chen, Junyang Lin, Yichang Zhang, Hongxia Yang, Jingren Zhou, and Jie Tang. 2019. Towards knowledge-based personalized product description generation in e-commerce. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3040–3050.

Digital Library

[7]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.

[8]

Ondřej Dušek and Filip Jurčíček. 2016. Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. arXiv preprint arXiv:1606.05491(2016).

[9]

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 1243–1252.

Digital Library

[10]

Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1602–1613.

[11]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672–2680.

[12]

Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Yong Yu, and Jun Wang. 2018. Long Text Generation via Adversarial Training with Leaked Information. In AAAI.

[13]

Joseph L Hellerstein, Yixin Diao, Sujay Parekh, and Dawn M Tilbury. 2004. Feedback control of computing systems. John Wiley & Sons.

[14]

Eduard H Hovy, Chin-Yew Lin, Liang Zhou, and Junichi Fukumoto. 2006. Automated Summarization Evaluation with Basic Elements. In LREC, Vol. 6. Citeseer, 604–611.

[15]

Zhiting Hu, Haoran Shi, Zichao Yang, Bowen Tan, Tiancheng Zhao, Junxian He, Wentao Wang, Xingjiang Yu, Lianhui Qin, Di Wang, 2018. Texar: A modularized, versatile, and extensible toolkit for text generation. arXiv preprint arXiv:1809.00794(2018).

[16]

Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. 2017. Toward controlled generation of text. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 1587–1596.

Digital Library

[17]

Oleg Ivanov, Michael Figurnov, and Dmitry Vetrov. 2018. Variational Autoencoder with Arbitrary Conditioning. arXiv preprint arXiv:1806.02382(2018).

[18]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114(2013).

[19]

Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547(2017).

[20]

Pan Li and Alexander Tuzhilin. 2019. Towards Controllable and Personalized Review Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 3228–3236.

[21]

Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming-Ting Sun. 2017. Adversarial ranking for language generation. In Advances in Neural Information Processing Systems. 3155–3165.

[22]

Zachary C Lipton, Sharad Vikram, and Julian McAuley. 2015. Capturing meaning in product reviews with character-level generative text models. arXiv preprint arXiv:1511.03683(2015).

[23]

Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In Advances in neural information processing systems. 700–708.

[24]

Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin, 2019. Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing. arXiv preprint arXiv:1903.10145(2019).

[25]

Jianmo Ni and Julian McAuley. 2018. Personalized review generation by expanding phrases and attending on aspect-aware representations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 706–711.

[26]

Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. 2018. Towards controllable story generation. In Proceedings of the First Workshop on Storytelling. 43–49.

[27]

Youbin Peng, Damir Vrancic, and Raymond Hanus. 1996. Anti-windup, bumpless, and conditioned transfer techniques for PID controllers. IEEE Control systems magazine 16, 4 (1996), 48–57.

[28]

Stanislau Semeniuta, Aliaksei Severyn, and Erhardt Barth. 2017. A hybrid convolutional variational autoencoder for text generation. arXiv preprint arXiv:1702.02390(2017).

[29]

Xiaoyu Shen, Hui Su, Shuzi Niu, and Vera Demberg. 2018. Improving variational encoder-decoders in dialogue generation. In Thirty-Second AAAI Conference on Artificial Intelligence.

[30]

Guillermo J Silva, Aniruddha Datta, and Shankar P Bhattacharyya. 2003. On the stability and controller robustness of some popular PID tuning rules. IEEE Trans. Automat. Control 48, 9 (2003), 1638–1641.

[31]

I Sutskever, O Vinyals, and QV Le. 2014. Sequence to sequence learning with neural networks. Advances in NIPS (2014).

[32]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008.

[33]

Jinpeng Wang, Yutai Hou, Jing Liu, Yunbo Cao, and Chin-Yew Lin. 2017. A statistical framework for product description generation. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). 187–192.

[34]

Qingyun Wang, Qi Zeng, Lifu Huang, Kevin Knight, Heng Ji, and Nazneen Fatema Rajani. 2020. ReviewRobot: Explainable Paper Review Generation based on Knowledge Synthesis. arXiv preprint arXiv:2010.06119(2020).

[35]

Wenlin Wang, Zhe Gan, Hongteng Xu, Ruiyi Zhang, Guoyin Wang, Dinghan Shen, Changyou Chen, and Lawrence Carin. 2019. Topic-Guided Variational Autoencoders for Text Generation. arXiv preprint arXiv:1903.07137(2019).

[36]

Jingjing Xu, Xuancheng Ren, Junyang Lin, and Xu Sun. 2018. Diversity-promoting gan: A cross-entropy based generative adversarial network for diversified text generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 3940–3949.

[37]

Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. Seqgan: Sequence generative adversarial nets with policy gradient. In Thirty-First AAAI Conference on Artificial Intelligence.

Digital Library

[38]

Hongyu Zang and Xiaojun Wan. 2017. Towards automatic generation of product reviews from aspect-sentiment scores. In Proceedings of the 10th International Conference on Natural Language Generation. 168–177.

[39]

Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018. Generating informative and diverse conversational responses via adversarial information maximization. In Advances in Neural Information Processing Systems. 1810–1820.

[40]

Yizhe Zhang, Zhe Gan, and Lawrence Carin. 2016. Generating text via adversarial training. In NIPS workshop on Adversarial Training, Vol. 21.

[41]

Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. arXiv preprint arXiv:1703.10960(2017).

[42]

Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 1097–1100.

Digital Library

Cited By

Song YDing JYuan JLiao QLi Y(2024)Controllable Human Trajectory Generation Using Profile-Guided Latent DiffusionACM Transactions on Knowledge Discovery from Data10.1145/370173619:1(1-25)Online publication date: 25-Oct-2024
https://dl.acm.org/doi/10.1145/3701736
Wang SDu YGuo XPan BQin ZZhao L(2024)Controllable Data Generation by Deep Learning: A ReviewACM Computing Surveys10.1145/364860956:9(1-38)Online publication date: 25-Apr-2024
https://dl.acm.org/doi/10.1145/3648609
Santosh KKholmukhamedov TKumar MAarif MMuda IBala B(2024)Leveraging GPT-4 Capabilities for Developing Context-Aware, Personalized Chatbot Interfaces in E-commerce Customer Support Systems2024 10th International Conference on Communication and Signal Processing (ICCSP)10.1109/ICCSP60870.2024.10544016(1135-1140)Online publication date: 12-Apr-2024
https://doi.org/10.1109/ICCSP60870.2024.10544016
Show More Cited By

Recommendations

Editable User Profiles for Controllable Text Recommendations
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Methods for making high-quality recommendations often rely on learning latent representations from interaction data. These methods, while performant, do not provide ready mechanisms for users to control the recommendation they receive. Our work tackles ...
Computational techniques for more accurate and diverse recommendations
Keywords Generation Improves E-Commerce Session-based Recommendation
WWW '20: Proceedings of The Web Conference 2020

By exploring fine-grained user behaviors, session-based recommendation predicts a user’s next action from short-term behavior sessions. Most of previous work learns about a user’s implicit behavior by merely taking the last click action as the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
443
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)6

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Song YDing JYuan JLiao QLi Y(2024)Controllable Human Trajectory Generation Using Profile-Guided Latent DiffusionACM Transactions on Knowledge Discovery from Data10.1145/370173619:1(1-25)Online publication date: 25-Oct-2024
https://dl.acm.org/doi/10.1145/3701736
Wang SDu YGuo XPan BQin ZZhao L(2024)Controllable Data Generation by Deep Learning: A ReviewACM Computing Surveys10.1145/364860956:9(1-38)Online publication date: 25-Apr-2024
https://dl.acm.org/doi/10.1145/3648609
Santosh KKholmukhamedov TKumar MAarif MMuda IBala B(2024)Leveraging GPT-4 Capabilities for Developing Context-Aware, Personalized Chatbot Interfaces in E-commerce Customer Support Systems2024 10th International Conference on Communication and Signal Processing (ICCSP)10.1109/ICCSP60870.2024.10544016(1135-1140)Online publication date: 12-Apr-2024
https://doi.org/10.1109/ICCSP60870.2024.10544016
Wu LLiu PYuan YLiu SZhang Y(2023)Context-aware style learning and content recovery networks for neural style transferInformation Processing and Management: an International Journal10.1016/j.ipm.2023.10326560:3Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1016/j.ipm.2023.103265
Kim NJo HLee JChoi JNam BJung Y(2022)An Intensity Controlled Review Dataset Construction for Automatic Review GenerationThe Journal of Korean Institute of Information Technology10.14801/jkiit.2022.20.2.15720:2(157-165)Online publication date: 28-Feb-2022
https://doi.org/10.14801/jkiit.2022.20.2.157
Fatima NImran AKastrati ZDaudpota SSoomro A(2022)A Systematic Literature Review on Text Generation Using Deep Neural Network ModelsIEEE Access10.1109/ACCESS.2022.317410810(53490-53503)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3174108
Yuan LWang JYu LZhang X(2022)Hierarchical template transformer for fine-grained sentiment controllable generationInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10304859:5Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.ipm.2022.103048
Sahoo RNaik VSingh SMalik S(2021)GANs and VAEs As Methods of Synthetic Data Generation and Augmentation to Enhance Heart Disease PredictionInternational Journal of Engineering and Advanced Technology10.35940/ijeat.B3263.121122111:2(17-23)Online publication date: 30-Dec-2021
https://doi.org/10.35940/ijeat.B3263.1211221

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents