Userneeds turned to efio as they needed help to fix critical performance issues. They experienced issues especially during peaks when collecting data from Denmark’s top three largest media houses. When the monitoring solutions were implemented, this exposed several other critical bottlenecks that needed fixing.

Userneeds enables organizations to uncover key insights from niche markets by providing access to online market research panels, which have been built and nurtured from the ground up. Collecting end-user analytics data from several of the top-10 Danish websites, Userneeds hosts web services handling over 10 million requests per day.

The project

When Userneeds suffered from outages on a business-critical API, they ask efio for assistance. efio helped to implement logging and monitoring solutions based on Metric Beats, Elasticsearch, and Kibana to the existing platform. Once in place, efio quickly identified a series of bottlenecks that affected the performance.

Starting with the most urgent issues, efio adjusted Apache, PHP, and Laravel configurations. Next, we fine-tuned the Kibana dashboards for specific APIs,  implemented them for monitoring and logging of several applications and infrastructural components. Fortunately, this implementation exposed several infrastructural and software bottlenecks.

Finally, efio assisted during an upgrade of Gitlab’s major version. This caused backward compatibility issues with Helm and managing Amazon EKS while tweaking Gitlab worker cache.

While the platform was continuously strengthened, Userneeds was able to increase the traffic load on the platform.

Responsibilities and accomplishments:

• Troubleshooting multiple issues by including Docker, PHP opcache settings, Apache, CI/CD deployment strategies and solving initial peak performance issues, isolating and advised on the remainder of bottlenecks in relation to infrastructure and software.
• Designing asynchronous web service using AWS API Gateway and SQS to reduce peak load on web services.
• Troubleshooting and fixing Gitlab AutoDevops CI/CD including self-hosted Gitlab Runner configuration/infrastructure, Kubernetes integration, and Helm version upgrades.
• Setting up monitoring of Kubernetes (Amazon EKS) and dashboards in CloudWatch and Kibana using the Elastic Stack and Terraform.
• Application/Laravel logging with Elastic Common Schema into Elastic Search
• Advising on AWS best practices

efio’s role

efio’s consultant joined a team of three engineers and contributed to the role of Senior DevOps engineer.  

Tools and techniques used:

AWS, SQS, Elastic Search, Kibana, CloudWatch, Docker, Filebeat, Metricbeat, Kubernetes, EKS, Terraform, Helm, PHP, Laravel, Gitlab