--- title: "InfraSight: replacing SaaS monitoring with a fleet-wide observability and security platform" description: "A self-hosted monitoring and security platform built from the ground up for a managed hosting fleet: per-second stack-aware metrics, centralised security event visibility, fleet-wide software inventory, and CVE response in seconds instead of SSH sessions. Designed, built and still actively developed." url: "https://joshdargie.com/work/infrasight" site: Josh Dargie --- - **Role:** Product owner, architect, lead developer - **Scale:** Multi-datacentre fleet, 100,000+ websites - **Replaced:** Per-server SaaS monitoring plus a patchwork of scripts - **Status:** In production, ongoing development ## What the customer needed The client is a managed hosting provider running a fleet of servers across multiple datacentres, hosting more than 100,000 websites for customers in over 90 countries. Every server runs the same dense, interdependent hosting stack (control panel, containerised accounts, web server, database, mail, caching, search), and every one of those services produces metrics, logs and security events that have to be watched, correlated and acted on. For years that job fell to a third-party SaaS monitoring product plus internal scripts. It had stopped being adequate on four fronts at once. It only saw generic system metrics, so it could report a server at 40% CPU while one customer's runaway cron saturated their container and degraded everyone else on the box. It had its own outages, which meant the monitoring went dark at exactly the moments it was needed. Its per-server pricing grew linearly with the fleet and gave nothing back for scale. And all of the data lived on the vendor's infrastructure, on the vendor's retention terms, so answering a customer's "what happened last Thursday afternoon" depended on someone else still having the data. Security made it urgent. With vulnerability disclosures heading toward 66,000 a year, roughly 180 a day, every CVE became a fleet-management question: which servers, which version, have we patched? Without a central inventory the answer started with manual SSH sessions. And each server handled hostile traffic on its own, with no fleet-level view of a coordinated campaign. ## The plan The brief I set was a single platform, entirely self-hosted, that could see every layer of the stack on every server in real time, and that treated security events as first-class data alongside performance metrics rather than as a separate product. No SaaS dependency, no per-server licence, and nothing leaving the client's infrastructure. I designed it in phases so that it paid for itself early. Phase one replaced the SaaS product like for like: lightweight agents on every server shipping per-second system metrics to a central ingest API and a ClickHouse time-series store, with a fleet dashboard and per-server drill-downs. Phase two added the stack-aware collectors that the old tooling could never provide: resource usage per customer account, request rates per virtual host, database throughput per database, health of every supporting service, and process-level history so "what was eating resources at 3 a.m." has an answer. Phase three turned it into a security platform: centralised firewall event visibility across the fleet, an inventory agent that records every package version and account configuration fleet-wide, and external probes for uptime, TTFB, SSL, DNS and blocklists. Data design was the part that made the rest possible. Metrics are kept at per-second resolution for seven days, downsampled to minutely for ninety, and hourly beyond that, with queries selecting the right tier automatically. Payload limits, rate awareness and ingest telemetry protect the pipeline from its own success. The platform itself runs on isolated service tiers with a deliberately small external surface, centrally managed agent authentication, and a build pipeline kept separate from production. ## What it delivered The question "which customer is causing the load?" now has an answer in one click, because the platform understands accounts, virtual hosts and databases rather than just CPU percentages. A new CVE can be matched against the live software inventory in seconds, showing every affected server, whether the vulnerable service is exposed, and remediation progress as patches land. A brute-force campaign hitting fifty servers shows up as one campaign, with source geography and the option of a fleet-wide response, instead of fifty independent local events. Operationally, monitoring stopped having outages of its own, the recurring per-server subscription disappeared in favour of a fixed infrastructure cost that scales with hardware rather than licensing, and the company owns every byte of its telemetry with retention on its own terms. Operations engineers work from an interface built for information density: every chart interactive, every metric queryable across the full retention window, role-based access so viewers monitor, operators act and administrators configure. ## Ongoing commitment InfraSight is not a project that shipped and stopped; it is a product with a roadmap and a release cadence, and I continue to own it. Current work centres on anomaly detection and alerting on top of the metric store so the platform raises problems before customers do, capacity forecasting from the long-retention tiers, deeper automation of CVE remediation from identification through to verified patch, and expanding the collector library as the hosting stack evolves. Each release goes through the same build pipeline and the same isolation model as the first one. The broader lesson is the one I bring to every client: generic tools have a ceiling, and for a business whose infrastructure is the product, the right monitoring is the one that understands the stack. Sometimes that means buying; sometimes, at the right scale and with the right team, it means building. ## Stack - Custom lightweight agents (metrics, inventory, log shipping) - ClickHouse for time-series at three resolution tiers - Relational store for fleet inventory and configuration state - Structured log streaming - Host firewall event capture - Isolated service tiers for ingest, web, workers and data - Role-based access control and a separate build pipeline