In mid-August, Datadog announced the extension of its database monitoring service to PostgreSQL, MySQL, and SQL Server on the Azure cloud.
Launched a year ago, this service called Database Monitoring should enrich the observability specialist’s APM platform. After installing the Datadog agent, database administrators (DBAs) and DevOps teams can not only track typical database performance metrics (latencies, CPU consumption, disk usage, RAM , etc.), but especially to study the performance of queries. The Query Metrics view should thus make it possible to identify the slowest queries, to filter them and to know the number of rows updated or restored.
With Query Samples, it is then possible to compare queries, taking as reference those that represent the expected standard. This sampling should make it possible to identify “abnormalities in execution times and costs” of requests, as well as to attribute them to a user, an application or a client host.
Finally, the execution plans aim to detail how a database executes a query at a time T and its evolution over time. For example, the tool detects the successive activation of join algorithms (hash join), or aggregations. It is used to associate a start-up and execution cost with these operations and to consult the number of lines concerned. It is also possible to configure alerts in order to be notified when the execution of queries is abnormally long. However, execution plans only support SELECT, UPDATE, INSERT, DELETE, and REPLACE queries.
Database Monitoring, compatible with DBaaS SQL Azure, GCP and AWS
“Database monitoring gives a view of all incoming calls, aggregation and inventory of normalized request types, their typical performance. We suggest performance improvements regarding the database configuration,” summarizes Renaud Boutet, SVP Product Management at Datadog. “If a normalized query performs poorly, it’s often that an index needs to be reworked.”
Until then, the New York publisher offered support for these DBMSs on AWS and Google. Specifically, it supported Amazon RDS (for Postgres, MySQL, and SQL Server), Aurora (MySQL and Postgres), and Google Cloud SQL (MySQL, Postgres, and SQL Server). Users can also use Database monitoring to monitor these three DBMSs when they host them themselves.
“Database Monitoring is experiencing very strong growth,” says Renaud Boutet. “Now we expect to cover the three database platforms most used by our customers.” It was already possible to instrument the DBMS with the observability platform, but this work remained the responsibility of the customers.
In August 2021, the editor had started supporting self-hosted versions of PostgreSQL and MySQL. According to Renaud Boutet, Datadog took about six months to instrument Microsoft SQL Server. As a reminder, this proprietary database has its own query language: T-SQL. In addition, Datadog has adapted to infrastructures, first those of AWS, then GCP and finally Azure.
“From a point of volume of use, after PostgreSQL and MySQL, Microsoft SQL Server was the most natural base to instrument”, advances Renaud Boutet. “Furthermore, Microsoft is an ecosystem in its own right. It is also a way for us to provide more and more tools to customers who swear by Azure products”. In this case, Datadog and Microsoft provide an integration between the observability platform and Azure Monitor to collect all the metrics referenced there.
In and of itself, database observability on Datadog is nothing new. “As soon as Datadog was launched, we instrumented and detected the key metrics of a DBMS”, assures Renaud Boutet. “Then, with our APM tool, we could see the incoming calls and the performance perceived by the database consumer, but we didn’t have the execution plans to understand how the queries were executing.”
The database monitoring, a market that is moving out of its niche
Datadog isn’t alone in providing database monitoring tools. Historically, tools like Paessler PRTG, Nagios, Zabbix or Solarwinds are very often used by DBAs. The Redgate platform provides the same level of granularity as that of Datadog, while Zabbix or Solarwinds offer similar functions for NoSQL databases, MongoDB in mind. Conversely, if Datadog supports MongoDB monitoring, the performance analysis is much more manual.
“Our clients mainly use relational databases and I think that will remain true for a long time”, explains Renaud Boutet. “Datadog users deploy NoSQL databases for very specific uses”.
This does not mean that the publisher refuses to strengthen the analysis of the behavior of NoSQL DBMSs. “With Database Monitoring, we plan to support MongoDB, Elasticsearch, but also new data lakes such as Snowflake or Databricks”, anticipates the manager.
As the Datadog platform can be used by DBMS administrators and application engineers, this would facilitate dialogue between these two professions. They usually struggled to communicate to resolve performance issues. In any case, this is what Renaud Boutet says, who recalls a mantra from the publisher: collaboration.
However, the publisher is not alone in this niche. Dynatrace also has its database observability solution available to developers and DBAs. This does not require instrumentation for Cassandra, PostgreSQL, Couchdb, Couchbase, and Redis, while an extension is required for Oracle, MySQL, SAP HANA, and IBM db2. For now, Dynatrace provides analytics from incoming calls through JBDC, ADO.NET or PDO frameworks. Database-level instrumentation is only available in Early Access with Oracle RAC and AWS Oracle RDS.
For its part, New Relic integrates with DBMarlin, a tool offering features similar to those of Database Monitoring for IBM db2, CockroachDB, MariaDB, MySQL, Oracle, Postgres and SQL Server.