A crowd-AI dynamic neural network hyperparameter optimization approach for image-driven social sensing applications
References
[1]
Wang D., Szymanski B.K., Abdelzaher T., Ji H., Kaplan L., The age of social sensing, Computer 52 (1) (2019) 36–45.
[2]
Galesic M., Bruine de Bruin W., Dalege J., Feld S.L., Kreuter F., Olsson H., Prelec D., Stein D.L., van Der Does T., Human social sensing is an untapped resource for computational social science, Nature 595 (7866) (2021) 214–222.
[3]
Zhang P., Zhao Q., Gao J., Li W., Lu J., Urban street cleanliness assessment using mobile edge computing and deep learning, IEEE Access 7 (2019) 63550–63563.
[4]
Figueiras P., Herga Z., Guerreiro G., Rosa A., Costa R., Jardim-Gonçalves R., Real-time monitoring of road traffic using data stream mining, in: 2018 IEEE International Conference on Engineering, Technology and Innovation, (ICE/ITMC), IEEE, 2018, pp. 1–8.
[5]
Khajwal A.B., Noshadravan A., An uncertainty-aware framework for reliable disaster damage assessment via crowdsourcing, Int. J. Disaster Risk Reduct. 55 (2021).
[6]
Liu Y., Liu X., Gao S., Gong L., Kang C., Zhi Y., Chi G., Shi L., Social sensing: A new approach to understanding our socioeconomic environments, Ann. Assoc. Am. Geogr. 105 (3) (2015) 512–530.
[7]
Havinga I., Marcos D., Bogaart P.W., Hein L., Tuia D., Social media and deep learning capture the aesthetic quality of the landscape, Sci. Rep. 11 (1) (2021) 1–11.
[8]
Kankanamge N., Yigitcanlar T., Goonetilleke A., Kamruzzaman M., Can volunteer crowdsourcing reduce disaster risk? A systematic review of the literature, Int. J. Disaster Risk Reduct. 35 (2019).
[9]
Amin M.A., Ali A.A., Rahman A., et al., Visual attention-based comparative study on disaster detection from social media images, Innov. Syst. Softw. Eng. 16 (3) (2020) 309–319.
[10]
K.-A.C. Quan, V.-T. Nguyen, T.-C. Nguyen, T.V. Nguyen, M.-T. Tran, Flood level prediction via human pose estimation from social media images, in: Proceedings of the 2020 International Conference on Multimedia Retrieval, 2020, pp. 479–485.
[11]
Phan T.-T., Labhart F., Gatica-Perez D., My own private nightlife: understanding youth personal spaces from crowdsourced video, Proc. ACM Hum.-Comput. Interact. 3 (CSCW) (2019) 1–34.
[12]
Feurer M., Hutter F., Hyperparameter optimization, in: Automated Machine Learning, Springer, 2019, pp. 3–33.
[13]
Li X., Caragea D., Zhang H., Imran M., Localizing and quantifying damage in social media images, in: 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, (ASONAM), IEEE, 2018, pp. 194–201.
[14]
Fridman L., Terwilliger J., Jenik B., Deeptraffic: Crowdsourced hyperparameter tuning of deep reinforcement learning systems for multi-agent dense traffic navigation, 2018, arXiv preprint arXiv:1801.02805.
[15]
Shui C., Zhou F., Gagné C., Wang B., Deep active learning: Unified and principled method for query and training, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2020, pp. 1308–1318.
[16]
Hertel L., Collado J., Sadowski P., Ott J., Baldi P., Sherpa: Robust hyperparameter optimization for machine learning, SoftwareX 12 (2020).
[17]
Li Q., Zhang X., Li H., Online direction of arrival estimation based on deep learning, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, (ICASSP), IEEE, 2018, pp. 2616–2620.
[18]
Li Y., Ji R., Lin S., Zhang B., Yan C., Wu Y., Huang F., Shao L., Dynamic neural network decoupling, 2019, arXiv preprint arXiv:1906.01166.
[19]
Han Y., Huang G., Song S., Yang L., Wang H., Wang Y., Dynamic neural networks: A survey, IEEE Trans. Pattern Anal. Mach. Intell. 44 (11) (2021) 7436–7456.
[20]
Guerrero-Viu J., Hauns S., Izquierdo S., Miotto G., Schrodi S., Biedenkapp A., Elsken T., Deng D., Lindauer M., Hutter F., Bag of baselines for multi-objective joint neural architecture search and hyperparameter optimization, 2021, arXiv preprint arXiv:2105.01015.
[21]
Zhang D., Zhang Y., Li Q., Plummer T., Wang D., Crowdlearn: A crowd-ai hybrid system for deep learning-based damage assessment applications, in: 2019 IEEE 39th International Conference on Distributed Computing Systems, (ICDCS), IEEE, 2019, pp. 1221–1232.
[22]
Sener O., Savarese S., Active learning for convolutional neural networks: A core-set approach, 2017, arXiv preprint arXiv:1708.00489.
[23]
Zhang Y., Zong R., Shang L., Kou Z., Zeng H., Wang D., CrowdOptim: A crowd-driven neural network hyperparameter optimization approach to AI-based smart urban sensing, Proc. ACM Hum.-Comput. Interact, 6 (CSCW2) (2022) 1–27.
[24]
Samulowska M., Chmielewski S., Raczko E., Lupa M., Myszkowska D., Zagajewski B., Crowdsourcing without data bias: Building a quality assurance system for air pollution symptom mapping, ISPRS Int. J. Geo-Inf. 10 (2) (2021) 46.
[25]
Nallaperuma D., Nawaratne R., Bandaragoda T., Adikari A., Nguyen S., Kempitiya T., De Silva D., Alahakoon D., Pothuhera D., Online incremental machine learning platform for big data-driven smart traffic management, IEEE Trans. Intell. Transp. Syst. 20 (12) (2019) 4679–4690.
[26]
Wu D., Cui Y., Disaster early warning and damage assessment analysis using social media data and geo-location information, Decis. Support Syst. 111 (2018) 48–59.
[27]
Montella R., Kosta S., Foster I., DYNAMO: Distributed leisure yacht-carried sensor-network for atmosphere and marine data crowdsourcing applications, in: 2018 IEEE International Conference on Cloud Engineering, (IC2E), IEEE, 2018, pp. 333–339.
[28]
Bhatti S.S., Gao X., Chen G., General framework, opportunities and challenges for crowdsourcing techniques: A comprehensive survey, J. Syst. Softw. 167 (2020).
[29]
Falkner S., Klein A., Hutter F., BOHB: Robust and efficient hyperparameter optimization at scale, in: International Conference on Machine Learning, PMLR, 2018, pp. 1437–1446.
[30]
Li L., Jamieson K., Rostamizadeh A., Gonina E., Ben-tzur J., Hardt M., Recht B., Talwalkar A., A system for massively parallel hyperparameter tuning, Proc. Mach. Learn. Syst. 2 (2020) 230–246.
[31]
Bohdal O., Yang Y., Hospedales T., EvoGrad: Efficient gradient-based meta-learning and hyperparameter optimization, Adv. Neural Inf. Process. Syst. 34 (2021) 22234–22246.
[32]
Li L., Jamieson K., DeSalvo G., Rostamizadeh A., Talwalkar A., Hyperband: A novel bandit-based approach to hyperparameter optimization, J. Mach. Learn. Res. 18 (185) (2018) 1–52.
[33]
Pontes F.J., Amorim G., Balestrassi P.P., Paiva A., Ferreira J.R., Design of experiments and focused grid search for neural network parameter optimization, Neurocomputing 186 (2016) 22–34.
[34]
Zahedi L., Mohammadi F.G., Rezapour S., Ohland M.W., Amini M.H., Search algorithms for automated hyper-parameter tuning, 2021, arXiv preprint arXiv:2104.14677.
[35]
Maclaurin D., Duvenaud D., Adams R., Gradient-based hyperparameter optimization through reversible learning, in: International Conference on Machine Learning, PMLR, 2015, pp. 2113–2122.
[36]
Yuan Y., Wang W., Pang W., A genetic algorithm with tree-structured mutation for hyperparameter optimisation of graph neural networks, in: 2021 IEEE Congress on Evolutionary Computation, (CEC), IEEE, 2021, pp. 482–489.
[37]
Turner R., Eriksson D., McCourt M., Kiili J., Laaksonen E., Xu Z., Guyon I., Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020, 2021, arXiv preprint arXiv:2104.10201.
[38]
Wistuba M., Kadra A., Grabocka J., Dynamic and efficient gray-box hyperparameter optimization for deep learning, 2022, arXiv preprint arXiv:2202.09774.
[39]
Im D.J., Savin C., Cho K., Online hyperparameter optimization by real-time recurrent learning, 2021, arXiv preprint arXiv:2102.07813.
[40]
Dong X., Shen J., Wang W., Shao L., Ling H., Porikli F., Dynamical hyperparameter optimization via deep reinforcement learning in tracking, IEEE Trans. Pattern Anal. Mach. Intell. 43 (5) (2019) 1515–1529.
[41]
Mandel T., Best J., Tanaka R.H., Temple H., Haili C., Carter S.J., Schlechtinger K., Szeto R., Using the crowd to prevent harmful AI behavior, Proc. ACM Hum.-Comput. Interact, 4 (CSCW2) (2020) 1–25.
[42]
M. Kobayashi, K. Wakabayashi, A. Morishima, Human+ AI Crowd Task Assignment Considering Result Quality Requirements, in: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9, 2021, pp. 97–107.
[43]
A. Alshaibani, S. Carrell, L.-H. Tseng, J. Shin, A. Quinn, Privacy-preserving face redaction using crowdsourcing, in: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 8, 2020, pp. 13–22.
[44]
R. Yasmin, J.T. Grassel, M.M. Hassan, O. Fuentes, A.R. Escobedo, Enhancing Image Classification Capabilities of Crowdsourcing-Based Methods through Expanded Input Elicitation, in: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9, 2021, pp. 166–178.
[45]
Liakos P., Papakonstantinopoulou K., Ntoulas A., Delis A., Rapid detection of local communities in graph streams, IEEE Trans. Knowl. Data Eng. 34 (5) (2020) 2375–2386.
[46]
Zhou Y., Wang X., Guo S., Wen Y., He J., A cost-effective adaptive random testing algorithm for object-oriented software testing, J. Intell. Fuzzy Systems 41 (3) (2021) 4415–4423.
[47]
Feng W., Zhang C., Zhang W., Han J., Wang J., Aggarwal C., Huang J., Streamcube: Hierarchical spatio-temporal hashtag clustering for event exploration over the Twitter stream, in: 2015 IEEE 31st International Conference on Data Engineering, IEEE, 2015, pp. 1561–1572.
[48]
Tropp J.A., Yurtsever A., Udell M., Cevher V., Streaming low-rank matrix approximation with an application to scientific simulation, SIAM J. Sci. Comput. 41 (4) (2019) A2430–A2463.
[49]
Kumar P., Gupta A., Active learning query strategies for classification, regression, and clustering: A survey, J. Comput. Sci. Tech. 35 (2020) 913–945.
[50]
Ren P., Xiao Y., Chang X., Huang P.-Y., Li Z., Gupta B.B., Chen X., Wang X., A survey of deep active learning, ACM Comput. Surv. 54 (9) (2021) 1–40.
[51]
Yang Y., Ma Z., Nie F., Chang X., Hauptmann A.G., Multi-class active learning by uncertainty sampling with diversity maximization, Int. J. Comput. Vis. 113 (2015) 113–127.
[52]
M. Shukla, S. Ahmed, A mathematical analysis of learning loss for active learning in regression, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3320–3328.
[53]
Kee S., Del Castillo E., Runger G., Query-by-committee improvement with diversity and density in batch active learning, Inform. Sci. 454 (2018) 401–418.
[54]
Hino H., Eguchi S., Active learning by query by committee with robust divergences, Inf. Geom. (2022) 1–26.
[55]
Martins V.E., Cano A., Junior S.B., Meta-learning for dynamic tuning of active learning on stream classification, Pattern Recognit. 138 (2023).
[56]
Elsken T., Metzen J.H., Hutter F., Neural architecture search: A survey, J. Mach. Learn. Res. 20 (1) (2019) 1997–2017.
[57]
C. Liu, L.-C. Chen, F. Schroff, H. Adam, W. Hua, A.L. Yuille, L. Fei-Fei, Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 82–92.
[58]
Hung N.Q.V., Tam N.T., Tran L.N., Aberer K., An evaluation of aggregation techniques in crowdsourcing, in: International Conference on Web Information Systems Engineering, Springer, 2013, pp. 1–15.
[59]
Rolnick D., Veit A., Belongie S., Shavit N., Deep learning is robust to massive label noise, 2017, arXiv preprint arXiv:1705.10694.
[60]
Titterington D.M., Recursive parameter estimation using incomplete data, J. R. Stat. Soc. Ser. B Stat. Methodol. 46 (2) (1984) 257–267.
[61]
Jagannatham A.K., Rao B.D., Cramér-Rao lower bound for constrained complex parameters, IEEE Signal Process. Lett. 11 (11) (2004) 875–878.
[62]
Wang D., Kaplan L., Abdelzaher T., Aggarwal C.C., On scalability and robustness limitations of real and asymptotic confidence bounds in social sensing, in: 2012 9th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, (SECON), IEEE, 2012, pp. 506–514.
[63]
Twitter D., Twitter API v2, 2023, https://developer.twitter.com/en/docs/twitter-api.
[64]
D.T. Nguyen, F. Ofli, M. Imran, P. Mitra, Damage assessment from social media imagery data during disasters, in: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2017, pp. 569–576.
[65]
Elghaish F., Talebi S., Abdellatef E., Matarneh S.T., Hosseini M.R., Wu S., Mayouf M., Hajirasouli A., Nguyen T.-Q., Developing a new deep learning CNN model to detect and classify highway cracks, J. Eng., Des. Technol. 20 (4) (2022) 993–1014.
[66]
Shalaby E., ElShennawy N., Sarhan A., Utilizing deep learning models in CSI-based human activity recognition, Neural Comput. Appl. (2022) 1–18.
[67]
Camilleri M., Neri F., Papoutsidakis M., An algorithmic approach to parameter selection in machine learning using meta-optimization techniques, WSEAS Trans. Syst. 13 (1) (2014) 203–212.
[68]
McDuffie D., Using amazon’s mechanical turk: benefits, drawbacks, and suggestions, APS Observer 32 (2) (2019).
[69]
Banerjee M., Capozzoli M., McSweeney L., Sinha D., Beyond kappa: A review of interrater agreement measures, Canad. J. Statist. 27 (1) (1999) 3–23.
[70]
G. Huang, Z. Liu, L. Van Der Maaten, K.Q. Weinberger, Densely connected convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4700–4708.
[71]
Tan M., Le Q., Efficientnetv2: Smaller models and faster training, in: International Conference on Machine Learning, PMLR, 2021, pp. 10096–10106.
[72]
Zhang H., Wu C., Zhang Z., Zhu Y., Zhang Z., Lin H., Sun Y., He T., Muller J., Manmatha R., Li M., Smola A., ResNeSt: Split-attention networks, 2020, arXiv preprint arXiv:2004.08955.
[73]
Jarrett J., Saleh I., Blake M.B., Malcolm R., Thorpe S., Grandison T., Combining human and machine computing elements for analysis via crowdsourcing, in: 10th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing, IEEE, 2014, pp. 312–321.
[74]
Awad N., Mallik N., Hutter F., Dehb: Evolutionary hyperband for scalable, robust and efficient hyperparameter optimization, 2021, arXiv preprint arXiv:2105.09821.
[75]
Schmucker R., Donini M., Zafar M.B., Salinas D., Archambeau C., Multi-objective asynchronous successive halving, 2021, arXiv preprint arXiv:2106.12639.
[76]
Chicco D., Jurman G., The advantages of the matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation, BMC Genom. 21 (1) (2020) 6.
[77]
Hutter F., Lücke J., Schmidt-Thieme L., Beyond manual tuning of hyperparameters, KI-Künstliche Intelligenz 29 (4) (2015) 329–337.
[78]
Scardapane S., Wang D., Randomness in neural networks: an overview, Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 7 (2) (2017).
Index Terms
- A crowd-AI dynamic neural network hyperparameter optimization approach for image-driven social sensing applications
Index terms have been assigned to the content through auto-classification.
Recommendations
CrowdOptim: A Crowd-driven Neural Network Hyperparameter Optimization Approach to AI-based Smart Urban Sensing
CSCWAI-based smart urban sensing (ASUS) has emerged as a scalable and pervasive application paradigm in smart city planning and management that aims to automatically assess the physical status of the urban environments by leveraging AI techniques and ...
Frugal Crowd Sensing for Bus Arrival Time Prediction in Developing Regions: Poster Abstract
IoTDI '17: Proceedings of the Second International Conference on Internet-of-Things Design and ImplementationThe design of crowd sensing applications that can supplement public transportation information systems have generally assumed availability of high-speed Internet connections coupled with high data sampling and gathering via data-hungry application ...
Bayesian Hyperparameter Optimization of Deep Neural Network Algorithms Based on Ant Colony Optimization
Document Analysis and Recognition – ICDAR 2021AbstractWithin this paper we proposed a new method named BayesACO, to improve the convolutional neural network based on neural architecture search with hyperparameters optimization. At its essence BayesACO in first side uses Ant Colony Optimization (ACO) ...
Comments
Information & Contributors
Information
Published In
Elsevier B.V.
Publisher
Elsevier Science Publishers B. V.
Netherlands
Publication History
Published: 25 October 2023
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Sep 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in