Existing Mobile Broadband Performance Measurements: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Catherine Yang and Version 1 by Vincent Umoh.

Globally, mobile broadband (MBB) penetration has increased due to the widespread use of smart devices, better mobile network coverage, and the ongoing quest for faster wireless and mobile communication technology. This has led to a tremendous rise in the number of internet subscribers, which is positively impacting the social and economic well-being of society at large. Terrestrial mobile network operators (MNOs) are responsible for providing MBB services to end users, but some of them do not offer the advertised speeds or theoretical speeds specified by 3GPP specifications. Therefore, periodic impartial and unbiased performance measurement studies of the quality of service (QoS) provided by the MNOs as perceived by the end users are required to help MNOs enhance the capabilities of their MBB networks and provide services at an acceptable quality.

  • measurement
  • methodology
  • mobile broadband

1. User-Centric Performance Evaluation Works

The research objective of most MBB studies is conducted to comparatively evaluate the MBB performance delivered by different MNOs. The research reported in [1][2] adopted a host and crowdsourced based approached using MBperf as the mobile application to measure the performance of 2G and 3G MBB networks, while [3] used a simplified Raspberry Pi testbed for measurement of the performance of 3G and 4G MBB networks over an extended period. The results of this research carried out in Nigeria reveal variations in MBB speeds delivered by four major MNOs in the country. Similarly, studies reported in [4][5][6] also used a panel-based crowdsourced approach for a comparative assessment of 3G and 4G MBB networks in Nepal, Pakistan and South Africa, respectively. They identified that the MBB speeds delivered to end users do not meet the values advertised by the MNOs. The behavior of these crowdsourced MBB measurement datasets can be analyzed using machine learning for more accurate estimations [7][8].
Apart from comparative analysis, other types of MBB performance evaluation have been carried out. For instance, [9] used a panel-based crowdsourced method for performance assessment of MBB services offered by different Internet service providers during defined peak periods and off-peak periods in major Canadian metropolitan areas. They defined peak periods as the time between 7 pm and 11 pm from Monday to Friday and off-peak periods as any hours or days exclusive of peak periods. Additionally, [10][11][12] used a dedicated testbed and drive tests approach to study the performance of different MBB networks under mobility, while [13][14] adopted the walk test methodology to perform coverage and capacity measurement and characterize the performance of MBB networks during peak periods and off-peak period.
Furthermore, when designing future technologies, MBB measurement can be valuable for benchmarking and planning network upgrades. The MBB performance measurement of the 4G networks reported in [12][15] are studies conducted to determine the baseline for 5G capabilities and assess the inefficiencies that should be addressed in the 5G network. Some of the points highlighted and the benchmarks estimated were considered in the 5G pilot MBB measurement reported in [16][17][18][19].

2. Testbed-Based Measurement Projects

The limitations posed by using the aforementioned methodologies have driven institutions and private researchers to develop more robust infrastructure for testbed-based experiments on MBB performance. Although some of these testbeds are expensive to build, they allow for a controlled and scalable measurement over a long period and thus, eliminate many limitations of the other methodologies. This section introduces some testbed-based MBB performance evaluation platforms and projects that already exist. It goes further to explain the network tools used for these testbeds.

2.1. The Nornet Edge (NNE) Platform

The Nornet Edge (NNE) platform is a testbed dedicated to the measurement and study of MBB networks and is presented in [20][21][22]. Figure 1 shows the overview of the testbed for MBB experiments. Renowned as one of the largest infrastructures in the world for MBB measurements, the NNE has over 400 fully programmable and multi-homed nodes shown in Figure 2, placed at different locations in Norway. The NNE measurement nodes comprise custom-made single-board computers running a standard Linux operating system that allow 2–5 MNOs to be connected to it using MBB modems. The node is equipped with a Samsung S5PV210 Cortex A8 1 GHz processor with 512 MB RAM, 512 MB NAND flash memory and a 16 GB SD card for storage. Sets of servers form a central backend system for collection and storing data on the NNE platform. There is also an algorithm designed to manage the nodes and run measurements for a long time on a national scale. The platform allows for the collection of status information from the modems on mobile broadband cell ID, connection mode and signal strength.
Figure 1.
The overall system architecture of the NNE platform
[20]
.
Figure 2.
NNE node with 4 modems connected
[20]
.
Since the platform is able to simultaneously connect to multiple networks, it is possible to directly compare QoS metrics across different MNOs. The NNE platform is built for future compatibility with new systems as its design makes it seamless to install new measurement applications to gather new or additional data. A website is also created for real-time viewing of the status of all NNE nodes, including the status of each MBB connection.
NNE is well suited for national scale measurements and experiments that require a large number of geographically distributed measurement nodes, simultaneous connections to multiple operators, information regarding the context in which measurements are taken, and continuous measurements that span long. One such experiment and research is reported in [10].

2.2. The MONROE Platform

The MONROE testbed and its operation presented in [23][24][25][26] is the first open access European transnational hardware-based platform for independent, multihomed, large-scale experimentation in MBB measurements. Figure 3 shows the overview of the MONROE MBB performance evaluation platform. MONROE has a set of 150 nodes, both mobile and stationary, which are multihomed to 5 different MNOs with the aid of commercial grade subscriptions across numerous European countries. The MONROE MBB measurement node shown in Figure 3 is based on Debian GNU/Linux “stretch” distribution integrating two small programmable computers. The computers are made of PC engines APU2 board interfacing with three 3G/4G MC7455 miniPCI express modems using LTE CAT6 and one WiFi modem. Each of the nodes gathers metadata such as carrier, technology, signal strength, GPS location and sensor data from the different modems. MONROE runs its MBB experiments using Docker containers (lightweight virtualized environment) to provide agile reconfiguration. Only users who are authenticated can access resources on the platform through a web portal, and also have access to the MONROE scheduler to deploy experiments. After each experiment on the MONROE platform, the results are periodically transferred from the nodes to a repository at a back-end server, while the MONROE scheduler also sets data quotas to ensure fairness among users. Some of the vast experiments run with the MONROE testbed have been reported in [27][28][29].
Figure 3.
Overview of the MONROE platform.
Three vital features of MONROE make the platform unique. It allows measurements to be repeated and controlled for precise and scientifically verifiable results for both fixed and mobile scenarios, enables support for demanding applications such as web and video services and supports protocol and service innovation.

2.3. The Simplified Raspberry Pi Platform

A simplified testbed for MBB performance evaluation that follows the setup of the NNE albeit using easily sourced commercial-off-the-shelf (COTS) devices is presented in [3][30]. Figure 4 shows the overall system architecture with the Raspberry Pi forming the core of the remote MBB measurement node. The Raspberry Pi 4 with 64 quad-core Cortex-A72 processors and 2GB Low-Power Double Data Rate (LPDDRA) RAM on its board is used for the node. The testbed uses USB modems and retrofitted WiFi to connect up to 4 MNOs for 3G and 4G MBB networks, respectively. The Raspberry Pi nodes are configured with the 4-way 5V relay modules mounted and an executable script written in python to achieve multihoming for 3G and 4G MBB measurements. The node autonomously initiates the measurement at regular intervals and stores the information, which an authorized user can access remotely at the testbed core for evaluation. This simplified MBB testbed is not as sophisticated as NNE; however, it can measure the key MBB performance metrics over an extended period.
Figure 4.
Overview of the Simplified Raspberry Pi platform
[30]
.
The aforementioned testbeds have been dedicated mostly to 3G and 4G MBB experiments, albeit allowing compatibility with future mobile communication networks like the recently deployed 5G network. To the best of ourthe knowledge, there is no dedicated testbed to assess the QoS delivered to end users on the 5G MBB network from a user-centric perspective. However, as part of the 5G Public Private Partnership (5G-PPP) initiative, the EU funded 5GENESIS [31] project has been developed as a flexible and open experimentation testbed for validating the end-to-end key performance indicators (KPIs) of 5G networks. The 5GENESIS architecture is designed to provide an integrated and open experimentation framework that facilitates interactions between the experimenters and the testing facilities. A detailed description of the experimentation suite is presented in [32], while pilot 5G experiments using the testbed have been reported in [33][34].
Furthermore, there are other testbed federations such as Fed4FIRE+ [35][36] and 5TONIC [37], developed to carry out experiments on numerous aspects of 4G and 5G. Fed4FIRE+ was the largest federation of internet testbeds in Europe consisting of 23 testbeds equipped with numerous user-friendly tools that enabled remote testing in different areas of interest. The Fed4FIRE+ project, which was a successor to the Fed4FIRE project, came to an end in June 2022 and its legacy will be taken by Scientific LargeScale Infrastructure for Computing/Communication Experimental Studies. (SLICES-RI) [35][36]. 5TONIC is an open research and innovation laboratory developed to create an open global environment for industry experts and members of academia to work together on specific projects that focus on 5G technologies [37]. Some studies that utilized the 5TONIC platform have been reported in [38][39].
Table 1 presents a summary of extensively reviewed user-centric MBB performance evaluation studies, highlighting the method adopted, the QoS metrics considered, the type of access network and a summary of each study.
Table 2 compares the different MBB performance measurement methods already discussed.
Table 1.
Summary of existing mobile broadband performance evaluation works.
S/N Refs. Methodology Adopted QoS Metrics Access Networks Study Summary
Features Drive Test Walk Test Crowdsourced Testbed
1 [1][2] Crowdsourced Download and upload throughput, latency and DNS lookup 2G and 3G Developed a mobile phone application (MBPerf) and adopted a host and crowdsourced based approach to carry out a comparative analysis of the performance of four MNOs that offer MBB services in a developing country.
2 [
Complexity Moderate Simple Moderate Complex
3][30] Testbed Download and upload throughput and latency 3G and 4G Used Raspberry Pi to develop a simplified testbed and conducted a comparative analysis of the MBB performance offered by four MNOs.
Accuracy Accurate Accurate (for the given purpose) Accurate (relying on users) Accurate 3 [9] Crowdsourced Download and upload speed, latency, packet loss and web loading time 4G Conducted MBB performance assessment during defined peak periods for Internet service providers using a panel-based crowdsourced method.
Scalability Moderate Low High Very High 4 [4][5] Crowdsourced Download and upload throughput, latency, packet loss, jitter and DNS resolution time 4G Provides a comparative investigation of MBB performance using a panel-based crowdsourced method.
Reliability and sensitivity Moderate Low Moderate High 5 [6] Crowdsourced Upload and download throughput and latency 4G Presented a comprehensive comparative study of user-centric MBB performance using a panel-based crowdsourced method.
Per measurement cost 6 [10] Testbed Latency, packet loss and connectivity 3G and 4G Studied the performance of MBB networks under mobility using a dedicated testbed for measurement
7 [40] Drive test Signal quality, downlink and uplink throughput, ping and handover 3G and 4G Developed and used a drive test method to evaluate and understand MBB performance in different locations
8 [41] Testbed Download speed 3G and 4G Presented a “speedtest like” measurement to estimate the download speed offered by MBB networks to users.
9 [42] Drive test Speed, coverage, satisfaction and latency 3G and 4G Conducted performance analysis of MBB networks to enable planning for 5G network upgrade in a rural area
10 [43] Testbed Web QoE, throughput, latency, and signal coverage 4G Examined the performance and response of nine mobile networks across Europe at different times during the COVID-19 pandemic
11 [44] Drive test Speed, coverage, satisfaction and latency 3G and 4G Conducted performance analysis of MBB networks to enable planning of 5G network upgrade urban area
12 [13] Walk test Received signal strength 3G and 4G Measured and characterized MBB performance through an indoor walk test
13 [45] Drive test Signal quality, Downlink and uplink throughput and Ping 3G and 4G Evaluated the MBB performance and coverage of existing MBB networks of different MNOs.
Proposed a supervise machine learning solution for a more accurate throughput estimation.
High High 14 [46] Drive test Throughput and latency 3G and 4G Measured real characteristics and coverage of MBB networks using a custom in-house made software tool.
15 [47] Testbed Latency and signal quality 4G Adapted data from extensive MBB measurement campaigns to develop a model suitable for realistic performance evaluation of applications and services.
16 [48] Drive test Signal strength 2G, 3G and 4G Determined and compared the signal strength of MBB networks of two MNOs.
17 [49] Drive test Throughput 2G, 3G and 4G Measured the performance of MBB networks using a custom mobile phone application.
Moderate 18 [11] Drive test Throughput and latency 3G and 4G Presented a comparative analysis of real MBB networks under mobility with expected theoretical expectations in order to identify the gaps between both.
Moderate 19 [50] Crowdsourced Throughput rates 4G Examined user perceived data rate fluctuations in 4G networks during different periods as well as compared the performance of MNOs
20 [14] Drive test and walk test Throughput and RTT 3G and 4G Performed coverage and capacity measurement of MBB networks in pedestrian zones during both busy hour and non-busy hour
21 [51] Drive test Signal quality and downlink and uplink throughput 4G Conducted performance analysis of 4G MBB networks and observed the propagation measurement of key performance indicators using drive test
22 [7] Crowdsourced and testbed Throughput 4G 23 [8] Crowdsourced Download and upload data rate, latency, signal strength 4G Developed a Machine Learning (ML) based framework used to define and determine different behavior of MNOs from crowdsourced datasets.
High (relative to the coverage) 24 [52] Crowdsourced Throughput, latency and DNS lookup 4G Conducted a longitudinal and multidimensional analysis of the extensive MBB measurement data collected to diagnose the cause behind observed performance variations.
25 [53] Testbed Latency 3G Examined delay characteristics in 3G networks from long-term MBB measurement data.
26 [54] Crowdsourced Latency 2G to 4G Analyzed latency of MBB networks from measurement data obtained using the crowdsourced method.
27 [55] Crowdsourced Throughput and latency 3G and 4G Used crowdsourced measurement data to study the characteristics of MBB network of three MNOs.
28 [56] Testbed Video streaming 4G Investigated the influence of different factors on YouTube streaming performance with different network configurations in four countries.
Conducted a full-fledged, end-to-end measurement study of the first commercial 5G networks.
Set up cost Moderate Low Very High 29 [57] Testbed Video streaming 4G Presented the design and implementation of a large-scale measurement tool for QoE when live streaming with MBB networks.
30 [58] Crowdsourced Throughput and latency of mobile apps 4G Extensively addressed the problem of QoE provisioning in smartphones from a double perspective, combining the results obtained from subjective laboratory tests with end-device passive MBB measurements and QoE crowd-sourced feedback obtained.
31 [16] Drive test Upload and download throughput, latency and packet loss 5G Conducted a pilot MBB measurement on the 5G network to investigate the QoS parameters of two MNOs.
32 [15] Crowdsourced Upload and download throughput, latency, jitter and packet loss 4G Performed a comparative assessment of the QoS parameters obtained from 4G MBB network and used it to establish a baseline for 5G MBB evaluation.
33 [17] Drive test Upload and download speed 5G Performed both stationary and mobility field tests to study the efficiency and performance of 5G networks.
34 [18] Drive test Upload and download speeds and latency 5G Conducted stationary field test to assess the MBB QoS performance of the 5G network using three popular mobile phone speedtest applications.
35 [59] Crowdsourced Download speed 4G Conducted a comparative study of download speeds on the 4G networks.
36 [60] Testbed Latency, jitter, packet loss, and throughput 3G Conducted a study to obtain the end-to-end parameters of the QoS for internet usage from a user perspective.
37 [12] Testbed Latency, upload throughput, handover 4G Measured the key MBB performance metrics of 4G networks under mobility, highlighting inefficiencies that need to be considered when designing the mobility features in 5G networks.
38 [61] Testbed Throughput, latency, jitter 4G and 5G Presents initial MBB measurement results of the key performance indicators on the 5G network.
39 [62] Walk Test Throughput, latency 4G and 5G Used private LTE and 5G networks to measure MBB performance metrics.
40 [19] Crowdsourced Handover, Mobile app performance 5G Performed extensive field tests of 5G network performance in different urban areas.
41 [63] Crowdsourced Throughput, latency, handover 5G Used a custom measurement tool to conduct a comprehensive measurement of numerous key aspects of commercial end-to-end 5G network performance.
42 [64] Testbed Throughput, latency, coverage 5G 43 [65] Testbed and Drive test Throughput, RTT, loss rate, signal quality 4G and 5G Performed a comparative study of the key performance metrics of 5G in extreme mobility.
44 [66] Drive test Throughput, latency, video streaming 5G Conducted an in-depth measurement study of 5G network performance in transit.
45 [67] Crowdsourced Throughput, latency 4G and 5G Performed a comparative study of 4G and 5G network deployment in two cities using a panel-based approach.
46 [33] Testbed Throughput, latency 5G Developed a modular and flexible experimentation methodology for validating 5G network KPIs.
Table 2.
Comparison of the different MBB measurement methods.

References

  1. Dahunsi, F.; Akinlabi, A. Measuring mobile broadband performance in Nigeria: 2G and 3G. Niger. J. Technol. 2019, 38, 422–436.
  2. Folasade, D.; Ayokunle, A.; Olumide, O.; Jide, P. Performance Monitoring of Mobile Broadband in a Developing Country. In Proceedings of the IST-Africa 2019 Conference, Nairobi, Kenya, 8–10 May 2019.
  3. Umoh, V.B.; Ukommi, U.S.; Ekpe, U.M. A Simplified Method for Extended Duration Measurements of Mobile Broadband Performance. In Proceedings of the 2022 IEEE Nigeria 4th International Conference on Disruptive Technologies for Sustaina-ble Development (NIGERCON), Lagos, Nigeria, 5–7 April 2022; pp. 1–5.
  4. Karn, N.K.; Hongli, Z.; Shafiq, M. Measuring broadband internet performance in Nepal: A comparative study. Procedia Comput. Sci. 2017, 107, 64–69.
  5. Awan, M.F.; Ahmad, T.; Qaisar, S.; Feamster, N.; Sundaresan, S. Measuring broadband access network performance in Pakistan: A comparative study. In Proceedings of the 2015 IEEE 40th Local Computer Networks Conference Workshops (LCN Workshops), Clearwater Beach, FL, USA, 26–29 October 2015; IEEE: New York, NY, USA, 2015; pp. 595–602.
  6. Chetty, M.; Sundaresan, S.; Muckaden, S.; Feamster, N.; Calandro, E. Measuring broadband performance in South Africa. In Proceedings of the 4th Annual Symposium on Computing for Development, Cape Town, South Africa, 6–7 December 2013; pp. 1–10.
  7. Kousias, K.; Alay, O.; Argyriou, A.; Lutu, A.; Riegler, M. Estimating downlink throughput from end-user measurements in mobile broadband networks. In Proceedings of the 2019 IEEE 20th International Symposium on A World of Wireless, Mobile and Multimedia Networks(WoWMoM), Washington, DC, USA, 10–12 June 2019; IEEE: New York, NY, USA, 2019; pp. 1–10.
  8. Kousias, K.; Midoglu, C.; Alay, O.; Lutu, A.; Argyriou, A.; Riegler, M. The same, only different: Contrasting mobile operator behavior from crowdsourced dataset. In Proceedings of the 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), Montreal, QC, Canada, 8–13 October 2017; IEEE: New York, NY, USA, 2017; pp. 1–6.
  9. Samknows. Analysis of Broadband Performance in Canada March & April 2016; The Canadian Radio-television and Telecommunications Commission (CRTC): Ottawa, ON, Canada, 2016.
  10. Siwakota, Y.R. Measuring Performance of Mobile Broadband Netwrok under Moblity; Department of Informatic, University of Oslo: Oslo, Norway, 2014.
  11. Ahmad, S.; Musleh, S.; Nordin, R. The gap between expectation & reality: Long term evolution (LTE) & third generation (3G) network performance in campus with test mobile system (TEMS). In Proceedings of the 2015 9th Asia Modelling Symposium (AMS), Kuala Lumpur, Malaysia, 7–9 September 2015; IEEE: New York, NY, USA, 2015; pp. 164–168.
  12. Parichehreh, A.; Moosavi, R.; Ramachandra, P.; Alfredsson, S.; Brunstrom, A. LTE as a Road Toward 5G: QoS Analysis in Mobility Scenario Using the Monroe Platform. In Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC), Marrakesh, Morocco, 15–18 April 2019; IEEE: New York, NY, USA, 2019; pp. 1–7.
  13. Shayea, I.; Ergen, M.; Azmi, M.H.; Nandi, D.; El-Salah, A.A.; Zahedi, A. Indoor network signal coverage of mobile telecommunication networks in West Malaysia: Selangor and Johor Bahru. In Proceedings of the 2017 IEEE 13th Malaysia International Conference on Communications (MICC), Johor Bahru, Malaysia, 28–30 November 2017; pp. 288–293.
  14. Turniski, F.; Lackovic, S.; Pilinsky, S.Z.; Tekovic, A. Analysis of 3G and 4G download throughput in pedestrian zones. In Proceedings of the 2016 International Symposium ELMAR, Zadar, Croatia, 12–14 September 2016; IEEE: New York, NY, USA, 2016; pp. 9–12.
  15. Daengsi, T.; Wuttidittachotti, P. Quality of Service as a Baseline for 5G: A Recent Study of 4G Network Performance in Thailand. In Proceedings of the 2020 IEEE International Conference on Communication, Networks and Satellite (Comnetsat), Batam, Indonesia, 17–18 December 2020; IEEE: New York, NY, USA, 2020; pp. 395–399.
  16. Daengsi, T.; Ungkap, P.; Wuttidittachotti, P. A Study of 5G Network Performance: A Pilot Field Trial at the Main Skytrain Stations in Bangkok. In Proceedings of the 2021 International Conference on Artificial Intelligence and Computer Science Technology (ICAICST), Yogyakarta, Indonesia, 29–30 June 2021; IEEE: New York, NY, USA, 2021; pp. 191–195.
  17. Daengsi, T.; Ungkap, P.; Wuttidittachotti, P. 5G Network Performance: A Study using Stationary and Mobility Tests on Sukhumvit Line–BTS Skytrain in Bangkok. In Proceedings of the 2021 4th International Conference of Computer and Informatics Engineering (IC2IE), Depok, Indonesia, 14–15 September 2021; IEEE: New York, NY, USA, 2021; pp. 447–450.
  18. Daengsi, T.; Ungkap, P.; Pornpongtechavanich, P.; Wuttidittachotti, P. QoS Measurement: A Comparative Study of Speeds and Latency for 5G Network Using Different Speed Test Applications for Mobile Phones. In Proceedings of the 2021 IEEE 7th International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA), Virtual, 23–25 August 2021; pp. 206–210.
  19. Narayanan, A.; Ramadan, E.; Carpenter, J.; Liu, Q.; Liu, Y.; Qian, F.; Zhang, Z.-L. A first look at commercial 5G performance on smartphones. Proc. Web Conf. 2020, 2020, 894–905.
  20. Kvalbein, A.; Baltrūnas, D.; Evensen, K.; Xiang, J.; Elmokashfi, A.; Ferlin-Oliveira, S. The Nornet Edge platform for mobile broadband measurements. Comput. Netw. 2014, 61, 88–101.
  21. Gran, E.G.; Dreibholz, T.; Kvalbein, A. NorNet Core–A multi-homed research testbed. Comput. Netw. 2014, 61, 75–87.
  22. Dreibholz, T.; Gran, E.G. Design and Implementation of the NORNET CORE Research Testbed for Multi-homed Systems. In Proceedings of the 2013 27th International Conference on Advanced Information Networking and Applications Workshops, Barcelona, Spain, 25–28 March 2013; pp. 1094–1100.
  23. Alay, O.; Lutu, A.; Garcia, R.; Peon-Quiros, M.; Mancuso, V.; Hirsch, T.; Dely, T.; Werme, J.; Evensen, K.; Hansen, A.; et al. Measuring and assessing mobile broadband networks with MONROE. In Proceedings of the 2016 IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM), Coimbra, Portugal, 21–24 June 2016; pp. 1–3.
  24. Alay, Ö.; Lutu, A.; Peón-Quirós, M.; Mancuso, V.; Hirsch, T.; Evensen, K.; Hansen, A.; Alfredsson, S.; Karlsson, J.; Brunstrom, A.; et al. Experience: An open platform for experimentation with commercial mobile broadband networks. In Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking, Snowbird, UT, USA, 16–20 October 2017; pp. 70–78.
  25. Alay, O.; Lutu, A.; García, R.; Peón-Quirós, M.; Mancuso, V.; Hirsch, T.; Dely, T.; Werme, J.; Evensen, K.; Hansen, A.; et al. MONROE, a distributed platform to measure and assess mobile broadband networks: Demo. In Proceedings of the Tenth ACM International Workshop on Wireless Network Testbeds, Experimental Evaluation, and Characterization, New York, NY, USA, 3–7 October 2016.
  26. Alay, O.; Lutu, A.; García, R.; Peón-Quirós, M.; Mancuso, V.; Hirsch, T.; Dely, T.; Werme, J.; Evensen, K.; Hansen, A.; et al. MONROE: Measuring Mobile Broadband Networks in Europe. 2017. Available online: https://dspace.networks.imdea.org/handle/20.500.12761/947 (accessed on 1 January 2022).
  27. Mancuso, V.; Quirós, M.P.; Midoglu, C.; Moulay, M.; Comite, V.; Lutu, A.; Alay, Ö.; Alfredsson, S.; Rajiullah, M.; Brunström, A.; et al. Results from running an experiment as a service platform for mobile broadband networks in Europe. Comput. Commun. 2019, 133, 89–101.
  28. Midoglu, C.; Kousias, K.; Alay, Ö.; Lutu, A.; Argyriou, A.; Riegler, M.; Griwodz, C. Large scale speedtest experimentation in Mobile Broadband Networks. Comput. Netw. 2021, 184, 107629.
  29. Midoglu, C.; Wimmer, L.; Lutu, A.; Alay, Ö.; Griwodz, C. MONROE-Nettest: A configurable tool for dissecting speed measurements in mobile broadband networks. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Honolulu, HI, USA, 15–19 April 2018; IEEE: New York, NY, USA, 2018; pp. 342–347.
  30. Umoh, V.B.; Ukommi, U.S.; Ekpe, U.M. A Comparative Study of User Experienced Mobile Broadband Performance. Niger. J. Technol. 2022, 41, 560–568.
  31. 5GENESIS. Available online: https://5genesis.eu/ (accessed on 24 October 2022).
  32. 5GENESIS. Deliverable D2.4: Final Report on Facility Design and Experimentation Planning. 2020. Available online: https://5genesis.eu/wp-content/uploads/2020/07/5GENESIS_D2.4_v1.0.pdf (accessed on 1 January 2022).
  33. Zayas, A.D.; Caso, G.; Alay, Ö.; Merino, P.; Brunstrom, A.; Tsolkas, D.; Koumaras, H. A modular experimentation methodology for 5G deployments: The 5GENESIS approach. Sensors 2020, 20, 6652.
  34. Christopoulou, M.; Xilouris, G.; Sarlas, A.; Koumaras, H.; Kourtis, M.-A.; Anagnostopoulos, T. 5g experimentation: The experience of the athens 5genesis facility. In Proceedings of the 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM), Bordeaux, France, 18–20 May 2021; IEEE: New York, NY, USA, 2021; pp. 783–787.
  35. FED4FIRE+. Available online: https://www.fed4fire.eu/ (accessed on 14 January 2023).
  36. Demeester, P.; Van Daele, P.; Wauters, T.; Hrasnica, H. Fed4FIRE–The Largest Federation of Testbeds in Europe. In Building the Future Internet through FIRE; River Publishers: Denmark, The Netherlands, 2022; pp. 87–109.
  37. 5TONIC. Available online: https://www.5tonic.org/ (accessed on 14 January 2023).
  38. Nogales, B.; Vidal, I.; Lopez, D.R.; Rodriguez, J.; Garcia-Reinoso, J.; Azcorra, A. Design and deployment of an open management and orchestration platform for multi-site nfv experimentation. IEEE Commun. Mag. 2019, 57, 20–27.
  39. Azcorra, A. Advanced 5G trials with verticals in 5TONIC laboratory. In International Ericsson All Employees Meeting of R&D; Ericsson: Madrid, Spain, 2017.
  40. El-Saleh, A.A.; Alhammadi, A.; Shayea, I.; Alsharif, N.; Alzahrani, N.M.; Khalaf, O.I.; Aldhyani, T.H.H. Measuring and Assessing Performance of Mobile Broadband Networks and Future 5G Trends. Sustainability 2022, 14, 829. Available online: https://www.mdpi.com/2071-1050/14/2/829 (accessed on 1 January 2022).
  41. Khatouni, A.S.; Mellia, M.; Marsan, M.A.; Alfredsson, S.; Karlsson, J.; Brunstrom, A.; Alay, O.; Lutu, A.; Midoglu, C.; Mancuso, V. Speedtest-like measurements in 3g/4g networks: The monroe experience. In Proceedings of the 2017 29th International Teletraffic Congress (ITC 29), Genoa, Italy, 4–8 September 2017; IEEE: New York, NY, USA, 2017; pp. 169–177.
  42. Shayea, I.; Azmi, M.H.; Ergen, M.; El-Saleh, A.A.; Han, C.T.; Arsad, A.; Rahman, T.A.; Alhammadi, A.; Daradkeh, Y.I.; Nandi, D. Performance analysis of mobile broadband networks with 5g trends and beyond: Urban areas scope in Malaysia. IEEE Access 2021, 9, 90767–90794.
  43. Rajiullah, M.; Khatouni, A.S.; Midoglu, C.; Alay, Ö.; Brunstrom, A.; Griwodz, C. Mobile network performance during the COVID-19 outbreak from a testbed perspective. In Proceedings of the 14th International Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization, London, UK, 21 September 2020; pp. 110–117.
  44. Shayea, I.; Ergen, M.; Azmi, M.H.; Nandi, D.; El-Salah, A.A.; Zahedi, A. Performance Analysis of Mobile Broadband Networks With 5G Trends and Beyond: Rural Areas Scope in Malaysia. IEEE Access 2020, 8, 65211–65229.
  45. Al Jahdhami, M.A.; El-Saleh, A.; Alhammadi, A.; Shayea, I. Performance Analysis of Mobile Broadband Networks in Ibra City, Oman. In Proceedings of the 2021 International Conference on Artificial Intelligence and Big Data Analytics, Xi’an, China, 27–29 October 2021; pp. 1–6.
  46. Fresolone, F.; Kloibhofer, R.; Ralbovsky, A.; Farkas, P.; Rakus, M.; Palenik, T. Throughput and one-way latency measurements in a 3G/4G live-network hi-mobility uplink. In Proceedings of the 2016 39th International Conference on Telecommunications and Signal Processing (TSP), Vienna, Austria, 27–29 June 2016; IEEE: New York, NY, USA, 2016; pp. 44–49.
  47. Albaladejo, M.B.; Leith, D.J.; Manzoni, P. Measurement-based modelling of lte performance in dublin city. In Proceedings of the 2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), Valencia, Spain, 4–8 September 2016; IEEE: New York, NY, USA, 2016; pp. 1–6.
  48. Engiz, B.K.; Kurnaz, Ç. Comparison of Signal Strengths of 2G/3G/4G services on a University Campus. Int. J. Appl. Math. Electron. Comput. 2016, 4, 37–42.
  49. Koprivica, L.Đ.M.; Nešković, N.; Nešković, A. Experimental performance analysis of THE 2G/3G/4G public mobile network. In Proceedings of the 2016 24th Telecommunications Forum (TELFOR), Belgrade, Serbia, 22–23 November 2016; pp. 1–4.
  50. Skocir, P.; Katusic, D.; Novotni, I.; Bojic, I.; Jezic, G. Data rate fluctuations from user perspective in 4G mobile networks. In Proceedings of the 2014 22nd International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia, 17–19 September 2014; IEEE: New York, NY, USA, 2014; p. 180185.
  51. Imoize, A.; Adegbite, O. Measurements-based performance analysis of a 4G LTE network in and around shopping malls and campus environments in Lagos Nigeria. Arid. Zone J. Eng. Technol. Environ. 2018, 14, 208.
  52. Nikravesh, A.; Choffnes, D.R.; Katz-Bassett, E.; Mao, Z.M.; Welsh, M. Mobile Network Performance from User Devices: A Longitudinal, Multidimensional Analysis. In Passive and Active Measurement; Cham, M.F., Kuzmanovic, A., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2014; pp. 12–22.
  53. Elmokashfi, A.; Kvalbein, A.; Xiang, J.; Evensen, K.R. Characterizing delays in Norwegian 3G networks. In International Conference on Passive and Active Network Measurement; Springer: Berlin/Heidelberg, Germany, 2012; pp. 136–146.
  54. Wang, X.; Xu, C.; Jin, W.; Zhao, G. A First Look at Cellular Network Latency in China. In Communications and Networking; Chen, Q., Meng, W., Zhao, L., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2018; pp. 339–348.
  55. Xu, Y.; Wang, Z.; Leong, W.K.; Leong, B. An End-to-End Measurement Study of Modern Cellular Data Networks. In Passive and Active Measurement; Faloutsos, M., Kuzmanovic, A., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2014; pp. 34–45.
  56. Schwind, A.; Midoglu, C.; Alay, Ö.; Griwodz, C.; Wamser, F. Dissecting the performance of YouTube video streaming in mobile networks. Int. J. Netw. Manag. 2020, 30, e2058.
  57. Schwind, A.; Seufert, M.; Alay, Ö.; Casas, P.; Tran-Gia, P.; Wamser, F. Concept and implementation of video QoE measurements in a mobile broadband testbed. In Proceedings of the 2017 Network Traffic Measurement and Analysis Conference (TMA), Dublin, Ireland, 21–23 June 2017; IEEE: New York, NY, USA, 2017; pp. 1–6.
  58. Casas, P.; Seufert, M.; Wamser, F.; Gardlo, B.; Sackl, A.; Schatz, R. Next to you: Monitoring quality of experience in cellular networks from the end-devices. IEEE Trans. Netw. Serv. Manag. 2016, 13, 181–196.
  59. Daengsi, T.; Chatchalermpun, S.; Praneetpolgrang, P.; Wuttidittachotti, P. A study of 4G network performance in Thailand referring to download speed. In Proceedings of the 2020 IEEE 10th Symposium on Computer Applications & Industrial Electronics (ISCAIE), Penang, Malaysia, 18–19 April 2020; IEEE: New York, NY, USA, 2020; pp. 160–163.
  60. Budiman, E.; Wicaksono, O. Measuring quality of service for mobile internet services. In Proceedings of the 2016 2nd International Conference on Science in Information Technology (ICSITech), Balikpapan, Indonesia, 26–27 October 2016; IEEE: New York, NY, USA, 2016; pp. 300–305.
  61. Soós, G.; Ficzere, D.; Varga, P.; Szalay, Z. Practical 5G KPI measurement results on a non-standalone architecture. In Proceedings of the Noms 2020–2020 IEEE/IFIP Network Operations and Management Symposium, Budapest, Hungary, 20–24 April 2020; IEEE: New York, NY, USA, 2020; pp. 1–5.
  62. Makino, I.; Wang, Z.; Terai, J.; Miki, N. Throughput and Delay Performance Measurements in Multi-Floor Building Employing Private LTE. IEEE Access 2022, 10, 24288–24301.
  63. Narayanan, A.; Zhang, X.; Zhu, R.; Hassan, A.; Jin, S.; Zhu, X.; Zhang, X.; Rybkin, D.; Yang, Z.; Mao, Z.; et al. A variegated look at 5G in the wild: Performance, power, and QoE implications. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference, Virtual, 23–27 August 2021; pp. 610–625.
  64. Xu, D.; Zhou, A.; Zhang, X.; Wang, G.; Liu, X.; An, C.; Shi, Y.; Liu, L.; Ma, H. Understanding operational 5G: A first measurement study on its coverage, performance and energy consumption. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication, New York, NY, USA, 10–14 August 2020; pp. 479–494.
  65. Pan, Y.; Li, R.; Xu, C. The first 5G-LTE comparative study in extreme mobility. Proc. ACM Meas. Anal. Comput. Syst. 2022, 6, 1–22.
  66. Fiandrino, C.; Martínez-Villanueva, D.J.; Widmer, J. Uncovering 5G Performance on Public Transit Systems with an App-based Measurement Study. In Proceedings of the ACM/IEEE International Conference on Modelling, Analysis and Simulation of Wireless and Mobile Systems, Montreal, QC, Canada, 24 October 2022; pp. 65–73.
  67. Rochman, M.I.; Sathya, V.; Nunez, N.; Fernandez, D.; Ghosh, M.; Ibrahim, A.S.; Payne, W. A comparison study of cellular deployments in Chicago and Miami using apps on smartphones. In Proceedings of the 15th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization, New Orleans, LA, USA, 31 January–4 February 2022; pp. 61–68.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
ScholarVision Creations
Feedback