skip to main content
10.1145/3377813.3381350acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article
Open access

Piranha: reducing feature flag debt at uber

Published: 18 September 2020 Publication History

Abstract

Feature flags are commonly used in mobile app development and can introduce technical debt related to deleting their usage from the codebase. This can adversely affect the overall reliability of the apps and increase their maintenance complexity. Reducing this debt without imposing additional overheads on the developers necessitates the design of novel tools and automated workflows.
In this paper, we describe the design and implementation of Piranha, an automated code refactoring tool which is used to automatically generate differential revisions (a.k.a diffs) to delete code corresponding to stale feature flags. Piranha takes as input the name of the flag, expected treatment behavior, and the name of the flag's author. It analyzes the ASTs of the program to generate appropriate refactorings which are packaged into a diff. The diff is assigned to the author of the flag for further processing, who can land it after performing any additional refactorings.
We have implemented Piranha to delete code in Objective-C, Java, and Swift programs, and deployed it to handle stale flags in multiple Uber apps. We present our experiences with the deployment of Piranha from Dec 2017 to May 2019, including the following highlights: (a) generated code cleanup diffs for 1381 flags (17% of total flags), (b) 65% of the diffs landed without any changes, (c) over 85% of the generated diffs compile and pass tests successfully, (d) around 80% of the diffs affect more than one file, (e) developers process more than 88% of the generated diffs, (f) 75% of the generated diffs are processed within a week, and (g) Piranha diffs have been interacted with by ~200 developers across Uber.
Piranha is available as open source at https://github.com/uber/piranha.

References

[1]
Pekka Abrahamsson, Juhani Warsta, Mikko T. Siponen, and Jussi Ronkainen. 2003. New Directions on Agile Methods: A Comparative Analysis. In Proc. of the 25th Intl. Conf. on Software Engineering (Portland, Oregon) (ICSE '03). 244--254.
[2]
Bram Adams, Wolfgang De Meuter, Herman Tromp, and Ahmed E. Hassan. 2009. Can we refactor conditional compilation into aspects?. In Proceedings of the 8th International Conference on Aspect-Oriented Software Development, AOSD 2009, Charlottesville, Virginia, USA, March 2--6, 2009. 243--254.
[3]
Edward Aftandilian, Raluca Sauciuc, Siddharth Priya, and Sundaresan Krishnan. 2012. Building Useful Program Analysis Tools Using an Extensible Java Compiler. In 12th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2012, Riva del Garda, Italy, September 23--24, 2012. 14--23.
[4]
Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. 2006. Compilers: Principles, Techniques, and Tools (2Nd Edition). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
[5]
Eric Allman. 2012. Managing Technical Debt. Commun. ACM 55, 5 (May 2012), 50--55.
[6]
AndroidDeadCode 2018. Shrink, obfuscate, and optimize your app. https://developer.android.com/studio/build/shrink-code. Accessed: 2019-01-29.
[7]
Mikio Aoyama. 1998. Agile Software Process and Its Experience. In Proceedings of the 20th International Conference on Software Engineering (Kyoto, Japan) (ICSE '98). IEEE Computer Society, Washington, DC, USA, 3--12.
[8]
Nathaniel Ayewah and William Pugh. 2010. The Google FindBugs Fixit. In Proceedings of the 19th International Symposium on Software Testing and Analysis (Trento, Italy) (ISSTA '10). ACM, New York, NY, USA, 241--252.
[9]
Subarno Banerjee, Lazaro Clapp, and Manu Sridharan. 2019. NullAway: practical type-based null safety for Java. In Proc. of the ACM European Software Engineering Conf. and Symp. on the Foundations of Software Engineering, FSE '19. 740--750.
[10]
Rajkishore Barik, Manu Sridharan, Murali Krishna Ramanathan, and Milind Chabbi. 2019. Optimization of Swift Protocols. In Object Oriented Programming Systems Languages and Applications (Athens, Greece) (OOPSLA '19).
[11]
Len Bass, Ingo Weber, and Liming Zhu. 2015. DevOps: A Software Architect's Perspective. Addison-Wesley, New York. http://my.safaribooksonline.com/9780134049847
[12]
Ira D. Baxter and Michael Mehlich. 2001. Preprocessor Conditional Removal by Simple Partial Evaluation. In Proc. of the 8th Working Conf. on Reverse Engineering, WCRE '01. 281--290.
[13]
Ira D. Baxter, Christopher W. Pidgeon, and Michael Mehlich. 2004. DMS: Program Transformations for Practical Scalable Software Evolution. In 26th Intl. Conf. on Software Engineering (ICSE 2004). 625--634.
[14]
Nelly Bencomo, Svein O. Hallsteinsen, and Eduardo Santana de Almeida. 2012. A View of the Dynamic Software Product Line Landscape. IEEE Computer 45, 10 (2012), 36--41.
[15]
Christian Bird and Thomas Zimmermann. 2012. Assessing the Value of Branches with What-if Analysis. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (Cary, North Carolina) (FSE '12). ACM, New York, NY, USA, Article 45, 11 pages.
[16]
Jan Bosch. 2000. Design and use of software architectures: adopting and evolving a product-line approach. Pearson Education.
[17]
Yuriy Brun, Reid Holmes, Michael D. Ernst, and David Notkin. 2011. Proactive Detection of Collaboration Conflicts. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (Szeged, Hungary) (ESEC/FSE '11). ACM, New York, NY, USA, 168--178.
[18]
Yuriy Brun, Reid Holmes, Michael D. Ernst, and David Notkin. 2013. Early Detection of Collaboration Conflicts and Risks. IEEE Trans. Softw. Eng. 39, 10 (Oct. 2013), 1358--1375.
[19]
Rafael Capilla, Jan Bosch, Pablo Trinidad, Antonio Ruiz Cortés, and Mike Hinchey. 2014. An overview of Dynamic Software Product Line architectures and techniques: Observations from research and industry. Journal of Systems and Software 91 (2014), 3--23.
[20]
M. Cataldo and J. D. Herbsleb. 2011. Factors leading to integration failures in global feature-oriented development: an empirical analysis. In 2011 33rd International Conference on Software Engineering (ICSE). 161--170.
[21]
Nanyu Chen, Min Liu, and Ya Xu. 2019. How A/B Tests Could Go Wrong: Automatic Diagnosis of Invalid Online Experiments. In Proceedings of the 12th ACM Intl. Conference on Web Search and Data Mining (WSDM '19). 501--509.
[22]
ChromeRelease 2012. Chrome Release Cycle. https://www.slideshare.net/Jolicloud/chrome-release-cycle. Accessed: 2019-01-29.
[23]
Jürgen Cito, Philipp Leitner, Martin Rinard, and Harald C. Gall. 2019. Interactive Production Performance Feedback in the IDE. In Proc. of the 41st Intl. Conference on Software Engineering (Montreal, Quebec, Canada) (ICSE '19). 971--981.
[24]
Paul Clements and Linda Northrop. 2002. Software product lines. Addison-Wesley.
[25]
DeadCode 1981. A survey of data flow analysis techniques. https://www.clear.rice.edu/comp512/Lectures/Papers/1981-kennedy-survey-scan.pdf. Accessed: 2019-01-29.
[26]
Differential Revision 2019. Differential User Guide. https://secure.phabricator.com/book/phabricator/article/differential/. Accessed: 2019-09-20.
[27]
Paul Duvall, Stephen M. Matyas, and Andrew Glover.2007. Continuous Integration: Improving Software Quality and Reducing Risk (The Addison-Wesley Signature Series). Addison-Wesley Professional.
[28]
FBRelease 2014. Native mobile app releases (Move fast and ship mobile things). https://www.youtube.com/watch?v=Nffzkkdq7GM. Accessed: 2019-01-29.
[29]
FeatureBits 1981. Feature Bits: Enabling Flow Within and Across Teams. https://www.infoq.com/presentations/Feature-Bits/. Accessed: 2019-01-29.
[30]
FlickrRelease 2009. 10+ Deploys Der Day: Dev and Ops Cooperation at Flickr. https://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr. Accessed: 2019-01-29.
[31]
Fowler 2010. Feature Toggles (aka Feature Flags). https://martinfowler.com/articles/feature-toggles.html. Accessed: 2019-01-29.
[32]
GateKeeper 2012. Building and testing at Facebook. https://www.facebook.com/notes/facebook-engineering/building-and-testing-at-facebook/10151004157328920/. Accessed: 2019-01-29.
[33]
GoogleRelease 2014. Testing Engineering@Google The Release Process for Google's Chrome for iOS. https://www.youtube.com/watch?v=p9bEc6oC6vw. Accessed: 2019-01-29.
[34]
Svein O. Hallsteinsen, Mike Hinchey, Sooyong Park, and Klaus Schmid. 2008. Dynamic Software Product Lines. IEEE Computer 41, 4 (2008), 93--95.
[35]
Mike Hinchey, Sooyong Park, and Klaus Schmid. 2012. Building Dynamic Software Product Lines. IEEE Computer 45, 10 (2012), 22--26.
[36]
Rashina Hoda and James Noble. 2017. Becoming Agile: A Grounded Theory of Agile Transitions in Practice. In Proceedings of the 39th International Conference on Software Engineering (Buenos Aires, Argentina) (ICSE '17). 141--151.
[37]
Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation (1st ed.). Addison-Wesley Professional.
[38]
Neil Jones, Carsten Gomard, and Peter Sestoft. 1993. Partial Evaluation and Automatic Program Generation.
[39]
Christian Kastner. 2019. Feature Flags vs Configuration Options - Same Difference? https://www.cs.cmu.edu/~ckaestne/featureflags/. Accessed: 2020-01-09.
[40]
Christian Kästner, Sven Apel, and Martin Kuhlemann. 2009. A model of refactoring physically and virtually separated features. In Generative Programming and Component Engineering, 8th Intl. Conf., GPCE 2009. 157--166.
[41]
Knight Capital 2013. SEC Charges Knight Capital With Violations of Market Access Rule. https://www.sec.gov/news/press-release/2013-222. Accessed: 2019-09-20.
[42]
Benjamin Livshits, Manu Sridharan, Yannis Smaragdakis, Ondřej Lhoták, J. Nelson Amaral, Bor-Yuh Evan Chang, Samuel Z. Guyer, Uday P. Khedker, Anders Møller, and Dimitrios Vardoulakis. 2015. In Defense of Soundiness: A Manifesto. Commun. ACM 58, 2 (Jan. 2015), 44--46.
[43]
Mateusz Machalica, Alex Samylkin, Meredith Porth, and Satish Chandra. 2019. Predictive Test Selection. In Proc. of the 41st Intl. Conf. on Software Engineering: Software Engineering in Practice (ICSE-SEIP 19). 91--100.
[44]
Magnus Madsen, Ming-Ho Yee, and Ondřej Lhoták. 2016. From Datalog to Flix: A Declarative Language for Fixed Points on Lattices. In Proc. of the 37th ACM SIGPLAN Conf. on Programming Language Design and Implementation (PLDI '16). ACM, New York, NY, USA, 194--208.
[45]
NetFlixRelease 2013. Deploying the Netflix API. https://medium.com/netflix-techblog/deploying-the-netflix-api-79b6176cc3f0. Accessed: 2019-01-29.
[46]
Flemming Nielson, Hanne R. Nielson, and Chris Hankin. 2010. Principles of Program Analysis. Springer Publishing Company, Incorporated.
[47]
Phabricator Task2019. Phabricator/Project management. https://www.mediawiki.org/wiki/Phabricator/Project_management. Accessed: 2019-09-20.
[48]
Klaus Pohl, Günter Böckle, and Frank J van Der Linden. 2005. Software product line engineering: foundations, principles and techniques. Springer Science & Business Media.
[49]
Problem Details 2018. The Rise and Fall of Knight Capital: Buy High, Sell Low. Rinse and Repeat. https://medium.com/dataseries/the-rise-and-fall-of-knight-capital-buy-high-sell-low-rinse-and-repeat-ae17fae780f6. Accessed: 2019-09-20.
[50]
Pull Request 2019. About pull requests. https://help.github.com/en/articles/about-pull-requests. Accessed: 2019-09-20.
[51]
Akond Ashfaque Ur Rahman, Eric Helms, Laurie Williams, and Chris Parnin. 2015. Synthesizing Continuous Deployment Practices Used in Software Development. In Proceedings of the 2015 Agile Conference (AGILE '15). IEEE Computer Society, Washington, DC, USA, 1--10.
[52]
Md Tajmilur Rahman, Louis-Philippe Querel, Peter C. Rigby, and Bram Adams. 2016. Feature Toggles: Practitioner Practices and a Case Study. In Proc. of the 13th Intl. Conf. on Mining Software Repositories (Austin, Texas) (MSR '16). 201--211.
[53]
ReleaseEng 2015. Guest Editors' Introduction: The Practice and Future of Release Engineering. https://www.youtube.com/watch?v=O3cJQTZXAI8. Accessed: 2019-01-29.
[54]
Thomas Reps, Susan Horwitz, and Mooly Sagiv. 1995. Precise Interprocedural Dataflow Analysis via Graph Reachability. In Proceedings of the 22Nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (San Francisco, California, USA) (POPL '95). ACM, New York, NY, USA, 49--61.
[55]
E. Shihab, C. Bird, and T. Zimmermann. 2012. The effect of branching strategies on software quality. In Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. 301--310.
[56]
SwiftSyntax 2019. Github link to SwiftSyntax. https://github.com/apple/swift-syntax. Accessed: 2019-09-20.
[57]
TechDebt 2014. Feature Toggles are one of the Worst kinds of Technical Debt. https://dzone.com/articles/feature-toggles-are-one-worst. Accessed: 2019-01-29.
[58]
Thomas Thüm, Sven Apel, Christian Kästner, Ina Schaefer, and Gunter Saake. 2014. A Classification and Survey of Analysis Strategies for Software Product Lines. ACM Comput. Surv. 47, 1 (2014), 6:1--6:45.
[59]
Linda Torczon and Keith Cooper. 2011. Engineering A Compiler (2nd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
[60]
Yangyang Zhao, Alexander Serebrenik, Yuming Zhou, Vladimir Filkov, and Bogdan Vasilescu. 2017. The Impact of Continuous Integration on Other Software Development Practices: A Large-scale Empirical Study. In Proceedings of the 32Nd IEEE/ACM Intl. Conf. on Automated Software Engineering (Urbana-Champaign, IL, USA) (ASE 2017). 60--71.
[61]
Celal Ziftci and Jim Reardon. 2017. Who Broke the Build?: Automatically Identifying Changes That Induce Test Failures in Continuous Integration at Google Scale. In Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP '17). 113--122.

Cited By

View all

Index Terms

  1. Piranha: reducing feature flag debt at uber

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICSE-SEIP '20: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Practice
    June 2020
    258 pages
    ISBN:9781450371230
    DOI:10.1145/3377813
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Sponsors

    In-Cooperation

    • KIISE: Korean Institute of Information Scientists and Engineers
    • IEEE CS

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 September 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article

    Conference

    ICSE '20
    Sponsor:

    Upcoming Conference

    ICSE 2025

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)197
    • Downloads (Last 6 weeks)30
    Reflects downloads up to 05 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media