Performance testing real-time data in JavaScript
Performance testing real-time data in JavaScript
By Daan van Berkel and Bader Ali
5 min read
To be able to chose the right protocol for handling real-time data, this post shares with you how we, based on speed, CPU- and memory usage, setup and performed performance tests on different real-time data protocols.
- Authors
- Name
- Daan van Berkel
- Name
- Bader Ali
This developer blog is a result of Bader and Daan's graduation research.
The protocols and frameworks compared in this blog are Long-polling, Mercure, Ratchet, SocketIO, server-sent Events (with NodeJS and ReactPHP) and Websockets.
Setting up a testing environment
The different nature of each protocol made it inherently difficult to write a generic performance testing interface. Why a generic interface? Because it opened the possibility to treat each protocol the same. It made it possible to write a single test executioner and have it inject a different protocol under the hood. This executioner, also called test-runner, is the core piece of code that can run the performance test and export the result to a JSON file.
To run the tests we used different tools: Docker(-compose), Puppeteer and NodeJS. This whole process is started by a simulate.js script. This script can be run with the following command and arguments to configure the performance test.
Why
While looking at the diagram above you may ask yourself a few questions. Let’s walk through those together.
Q 1. Why is Docker used in the performance tests?
Some of the protocols require a server that the browser can connect to. Docker allows us to write a docker-compose file for each protocol and have the test-runner start the Docker containers. Docker containers allow us to monitor the CPU and memory usage using the Docker Stats API.
const dockerUp = (readyText) =>
new Promise(async (res) => {
const docker = spawn('docker-compose', ['up'])
docker.stdout.on('data', function (data) {
if (data.toString().includes(readyText)) {
res()
}
})
})
// example usage
await dockerUp('PHP server started on')
Example 1: Start docker-compose from NodeJS.
const http = require('http')
function dockerWatchStats(containerName, callback) {
http
.request(
{
socketPath: '/var/run/docker.sock',
path: `http://localhost/v1.41/containers/${containerName}/stats`,
},
(res) => {
res.setEncoding('utf8')
res.on('data', (data) => {
data = JSON.parse(data)
// See https://docs.docker.com/engine/api/v1.41/#operation/ContainerStats for formula
let cpuPercentage = calculateCpuPercentage(data)
let memUsage = calculateMemory(data)
callback({ cpuPercentage, memUsage })
})
}
)
.end()
}
// Example
dockerWatchStats('', (usage) => {
console.log(usage)
})
Example 2: Get CPU- and memory usage from Docker.
Q 2. Why is a single instance of a Puppeteer browser started first?
Unfortunately, Puppeteer does not provide an API for monitoring the CPU and memory usage for each separate tab. However, these values can be extracted by the system itself using the built-in monitoring tools. The built-in monitoring only allows for measuring an entire process not a single tab in a web browser. For that reason, we need to spin up a separate Puppeteer browser with a single client to measure its performance. Below is a code example of how to do this in NodeJS, using the pidusage package:
const pidusage = require('pidusage')
function browserWatchStats(pid, callback) {
setInterval(() => {
pidusage(pid, function (err, stats) {
if (stats == null) return
callback({
cpuPercentage: stats.cpu,
memoryUsageInByte: stats.memory,
})
})
}, 500)
}
browserWatchStats('', (usage) => {
console.log(usage)
})
Example 3: Get CPU- and memory usage from Puppeteer.
Q 3. How to communicate between Node.JS and all Puppeteer instances?
After the browser is done receiving all messages from the server, it needs to communicate this information back to the NodeJS application. This can be done using a hook in the Puppeteer page on the console event. It is possible to listen to `console.log` events from a page and get the results in a callback in NodeJS. A code example:
page.on('console', (msg) => console.log(msg._text))
Example 4: Pass console message through to NodeJS.
Q 4. Why not send the messages directly from the test-runner?
To keep the test fair we need to make sure that the usage in the test does not differ from how the protocol is used in a production environment. We also need to make sure that the connections between each test are the same. Let's take ReactPHP as an example. The test-runner is written in JavaScript so it has no direct way of communicating with PHP:
The left option is not possible because of language collision. We can combat this issue by writing an HTTP endpoint in ReactPHP that communicates to the SSE code to send a message to the Puppeteer clients. However, we do not face this problem when using a JavaScript protocol/library. Let's take SocketIO as an example:
Since SocketIO is also written in JavaScript we could import it in the test-runner and make use of it directly. This however would not be a fair comparison. To make it fair we have added an ExpressJs endpoint that the test-runner can ping to.
Caveat: browser connection limit
Each protocol has its pros and cons. While executing the performance tests we noticed that some of them were maxing out on connections. For example, Server-Sent Event on HTTP/1.1 can only handle 6 connections per browser. This can be combated by starting a new browser per 6 tabs for Server-Sent Event.
Results
As we mentioned earlier, the test runner can export the results to a JSON file. But what fun would it be if we stopped there? To get a real feel and understanding of the results we need to visualize the data. For this, we chose to create a web application using NodeJS, ChartJS, AlphineJS and TailwindCSS. In the image below you find the results for 100 clients each receiving 1000 messages. The results are the average of 10 tests. The Y axis represents the time in milliseconds it took between receiving the first and last message.
You can discover all results at https://iodigital-com.github.io/techhub-performance-testing-real-time-data-examples.
We hope that this blog post helps you with removing some roadblocks with performance testing.
Upcoming events
The Test Automation Meetup
PLEASE RSVP SO THAT WE KNOW HOW MUCH FOOD WE WILL NEED Test automation is a cornerstone of effective software development. It's about creating robust, predictable test suites that enhance quality and reliability. By diving into automation, you're architecting systems that ensure consistency and catch issues early. This expertise not only improves the development process but also broadens your skillset, making you a more versatile team member. Whether you're a developer looking to enhance your testing skills or a QA professional aiming to dive deeper into automation, RSVP for an evening of learning, delicious food, and the fusion of coding and quality assurance! 🚀🚀 18:00 – 🚪 Doors open to the public 18:15 – 🍕 Let’s eat 19:00 – 📢 First round of Talks 19:45 – 🍹 Small break 20:00 – 📢 Second round of Talks 20:45 – 🍻 Drinks 21:00 – 🙋♀️ See you next time? First Round of Talks: The Power of Cross-browser Component Testing - Clarke Verdel, SR. Front-end Developer at iO How can you use Component Testing to ensure consistency cross-browser? Second Round of Talks: Omg who wrote this **** code!? - Erwin Heitzman, SR. Test Automation Engineer at Rabobank How can tests help you and your team? Beyond the Unit Test - Christian Würthner, SR. Android Developer at iO How can you do advanced automated testing for, for instance, biometrics? RSVP now to secure your spot, and let's explore the fascinating world of test automation together!
| Coven of Wisdom - Amsterdam
Go to page for The Test Automation MeetupCoven of Wisdom - Herentals - Winter `24 edition
Worstelen jij en je team met automated testing en performance? Kom naar onze meetup waar ervaren sprekers hun inzichten en ervaringen delen over het bouwen van robuuste en efficiënte applicaties. Schrijf je in voor een avond vol kennis, heerlijk eten en een mix van creativiteit en technologie! 🚀 18:00 – 🚪 Deuren open 18:15 – 🍕 Food & drinks 19:00 – 📢 Talk 1 20:00 – 🍹 Kleine pauze 20:15 – 📢 Talk 2 21:00 – 🙋♀️ Drinks 22:00 – 🍻 Tot de volgende keer? Tijdens deze meetup gaan we dieper in op automated testing en performance. Onze sprekers delen heel wat praktische inzichten en ervaringen. Ze vertellen je hoe je effectieve geautomatiseerde tests kunt schrijven en onderhouden, en hoe je de prestaties van je applicatie kunt optimaliseren. Houd onze updates in de gaten voor meer informatie over de sprekers en hun specifieke onderwerpen. Over iO Wij zijn iO: een groeiend team van experts die end-to-end-diensten aanbieden voor communicatie en digitale transformatie. We denken groot en werken lokaal. Aan strategie, creatie, content, marketing en technologie. In nauwe samenwerking met onze klanten om hun merken te versterken, hun digitale systemen te verbeteren en hun toekomstbestendige groei veilig te stellen. We helpen klanten niet alleen hun zakelijke doelen te bereiken. Samen verkennen en benutten we de eindeloze mogelijkheden die markten in constante verandering bieden. De springplank voor die visie is talent. Onze campus is onze broedplaats voor innovatie, die een omgeving creëert die talent de ruimte en stimulans geeft die het nodig heeft om te ontkiemen, te ontwikkelen en te floreren. Want werken aan de infinite opportunities van morgen, dat doen we vandaag.
| Coven of Wisdom Herentals
Go to page for Coven of Wisdom - Herentals - Winter `24 editionMastering Event-Driven Design
PLEASE RSVP SO THAT WE KNOW HOW MUCH FOOD WE WILL NEED Are you and your team struggling with event-driven microservices? Join us for a meetup with Mehmet Akif Tütüncü, a senior software engineer, who has given multiple great talks so far and Allard Buijze founder of CTO and founder of AxonIQ, who built the fundaments of the Axon Framework. RSVP for an evening of learning, delicious food, and the fusion of creativity and tech! 🚀 18:00 – 🚪 Doors open to the public 18:15 – 🍕 Let’s eat 19:00 – 📢 Getting Your Axe On Event Sourcing with Axon Framework 20:00 – 🍹 Small break 20:15 – 📢 Event-Driven Microservices - Beyond the Fairy Tale 21:00 – 🙋♀️ drinks 22:00 – 🍻 See you next time? Details: Getting Your Axe On - Event Sourcing with Axon Framework In this presentation, we will explore the basics of event-driven architecture using Axon Framework. We'll start by explaining key concepts such as Event Sourcing and Command Query Responsibility Segregation (CQRS), and how they can improve the scalability and maintainability of modern applications. You will learn what Axon Framework is, how it simplifies implementing these patterns, and see hands-on examples of setting up a project with Axon Framework and Spring Boot. Whether you are new to these concepts or looking to understand them more, this session will provide practical insights and tools to help you build resilient and efficient applications. Event-Driven Microservices - Beyond the Fairy Tale Our applications need to be faster, better, bigger, smarter, and more enjoyable to meet our demanding end-users needs. In recent years, the way we build, run, and operate our software has changed significantly. We use scalable platforms to deploy and manage our applications. Instead of big monolithic deployment applications, we now deploy small, functionally consistent components as microservices. Problem. Solved. Right? Unfortunately, for most of us, microservices, and especially their event-driven variants, do not deliver on the beautiful, fairy-tale-like promises that surround them.In this session, Allard will share a different take on microservices. We will see that not much has changed in how we build software, which is why so many “microservices projects” fail nowadays. What lessons can we learn from concepts like DDD, CQRS, and Event Sourcing to help manage the complexity of our systems? He will also show how message-driven communication allows us to focus on finding the boundaries of functionally cohesive components, which we can evolve into microservices should the need arise.
| Coven of Wisdom - Utrecht
Go to page for Mastering Event-Driven Design