Making AI Securely Available to the Masses with HAProxy
More than 2 million requests per second.
354 certified security controls.
Representative Provider for ModelOps and Explainable AI.
About Modzy
Founded in 2019 after several years of development, Modzy is now one of the leading ModelOps and MLOps platforms in the AI sector. With a goal of building a world where people and machines working together outperform either working alone, their service enables clients to create and shift AI models from the lab into production at scale. Compatible with both traditional and cloud infrastructure, Modzy is gearing up for growth as their polling suggests an enormous surge in AI investment in the near future.
Results at a Glance
The Challenge
As a platform provider for the deployment of artificial intelligence, Modzy enables clients to move lab-grown AI services into production and then subsequently scale, monitor, and improve their operations. This filled a need in the machine learning services industry for a deployment structure that took the strain off client-side engineering, allowing customers to reap the benefits of artificial intelligence at scale without worrying about its deployment and upkeep.
After a round of seed funding and several years of creating the structure of their platform, the team at Modzy was looking to build a free testing system for potential clients to accompany the unveiling of their suite of products. However, as the head of engineering at Modzy Nathan Mellis well knew, exposing any infrastructure to the internet comes with inherent risks. And so the team set about to create a highly-available infrastructure with security as a top priority to begin exposing their platform to the world.
.png)
Nathan Mellis at HAProxyConf 2021
The Objectives
Nathan and the team at Modzy had a detailed set of requirements that would address their security needs. These included protecting user traffic with HTTPS, implementing a web application firewall, restricting access to certain geographic regions, and enabling rate limiting.
Nathan ultimately decided on a cluster of HAProxy Enterprise load balancers for the more sophisticated security options available. These included the Advanced WAF and its ability to support more complex rulesets, as well as other features like real-time blacklists for Botnet IPs and increased granularity in GeoIP filtering. In combination with the implementation of stick tables for rate-limiting and enhanced SSL with server name indication, the engineering team slept well in the knowledge that they had a secure face for their new platform.
Nathan also took advantage of HAProxy Enterprise’s unrivaled observability by using a Fluent Bit logging and metrics processor. This converted and exported the balancer’s logs to their chosen SIEM tool, providing the team with a clear window into the operations and alerts of the load balancing system.
The Solution
In order to deploy this system, the Modzy engineers decided to utilize an infrastructure-as-code approach to provision and manage their various network parts. The implementation of HashiCorp Packer to create an HAProxy Enterprise virtual machine image – with Fluent Bit, logging configuration and security plugins pre-loaded – alongside HashiCorp Terraform to generate fresh configuration files when needed, meant that system updates were seamless.
HashiCorp Terraform auto-generates these new files as new custom domain names are created for each customer, writing and saving them to an S3 bucket. The new settings are subsequently added to the load balancer by a cron sync, and by taking advantage of HAProxy Enterprise’s hitless reloads, the load balancer restarts itself without losing any connections, resulting in a fully self-sustaining and highly available network.
.png)
The Results
The Modzy team also took advantage of the extensive documentation and web tutorials provided alongside the HAProxy Enterprise suite of products to design the structure of their HAProxy Enterprise instances in an AWS-hosted private subnet based directly upon a design recommended by HAProxy Technologies’ support engineers. Nathan also found his dialogue with the support team to be extremely helpful in setting up their system, meaning a very successful free trial platform for their AI services. The result of which was a system that used the power of HAProxy Enterprise to provide security and observability to their service’s free tier ultimately used to convert customers to their enterprise offerings.
What HAProxy Enterprise Offers You
Whether you need superior observability, protection against web-based threats, or simply a load balancer that can integrate with a custom platform, HAProxy Enterprise is the best in its field. Having recently surpassed the mark of 2 Million HTTP requests per second, you can deliver your own applications at unrivaled speeds with HAProxy Enterprise. Contact us to learn further.
Interested to learn more about HAProxy use cases? Explore our Success Stories page.