In today's highly interconnected IT world, where providing your customers with accurate monitoring data in a form they can understand gets more critical by the day, we need ways to accommodate such needs rapidly.
Satisfying the Customer's Needs
Dynatrace, as a leading monitoring tool in most cases, accrues a vast amount of data and allows the creation of dashboards to analyze the data and provide customized views for various use cases. However, sometimes this is not enough, and other means will be needed to satisfy the wishes of your users fully. Here are some examples where we had to take additional measures:
Overview Dashboard: Level 2 Support wants a cockpit view with data from different sources, including Dynatrace.
Periodic Data Exports: A manager needs a weekly data export from Dynatrace in a specific format for Power BI use.
Highly Specific Formats: The user wants to have CPU, memory, and disk metrics in a table, while disk usage should be shown for one specific disk, and the table should contain links to Logs and Dynatrace for further analysis.
The Challenges of the API
For the above use cases, you will need to query the Dynatrace API to get the required data so you can format it the exact way your customer needs it. However, the Dynatrace API, as mighty as it is, provides various challenges that often require a program to be written to do something with the data. Some of the problems we ran into are:
Data Structure: The data structure returned by many API endpoints is often a bit more complex with nested values.
Different Formats: Different endpoints have different formats. Therefore, a solution cannot be implemented as a standard solution that works with every endpoint.
Mapping Column Arrays: Some endpoints return arrays with data per column that must be mapped into a table structure.
Pagination: There is no fetch-all in many cases, and you must send multiple queries to fetch all the data.
Limits: Some endpoints, like 5000 records for the selected timeframe, have hard limits. The only way we found to solve this is to step through the specified timeframe, e.g., in 1-hour steps, where less than 5000 records are returned.
Available Solutions
While there are commercial solutions on the market, like Grafana with its Dynatrace Plugin, which solves the above problems to quite some extent but not entirely, the only fast and flexible way we from Performetriks have found is provided by the entirely free and open-source tool called Engineered Monitoring Platform (“EMP”). EMP is a versatile dashboarding, monitoring, reporting, and data analytics solution with a powerful query language that lets you fetch data from various sources, transform the data to your needs, and define how the data should be displayed or reported.
Dynatrace Data shown in EMP
Below is one example of a table in EMP that combines host metrics from different Dynatrace environments, colors them in traffic light style based on custom thresholds, and provides various links to open the application, the log management tool, or the host details in Dynatrace.
Overview dashboards such as below have been created in EMP for various application operation teams, combining data from various sources, including Dynatrace:
The following screenshot shows an example of the Session Export Dashboard. It splits the name in the “application” column and creates its columns. The user can export all the data as CSV, JSON or XML. The same query used for the below dashboard has been set up in EMP to run weekly and export the data by either sending an email or storing the data on a shared drive.
While Dynatrace has a sophisticated view for advanced users that gives many details about what is going on in a session, it can be challenging for a regular user to analyze what is happening, when, and how long it took. We can fetch the session data for a specific session and show a simplified view that shows a Gantt Chart, similar to what we know from the browser developer tools. Here an example:
Querying Dynatrace with EMP
The EMP query language has a Unix-shell-like flavor, passing table-like data from one command to the next. This query language can get data from various sources like web APIs, databases or command line outputs. Data from different sources can be combined, and subqueries can be used to fetch additional data from other sources.
To give you a basic example of how EMP uses its query language to get the data from Dynatrace, the following gets the CPU usage for every host in a list.
Conclusion
EMP empowers everyone to fetch data from Dynatrace, transform it as needed, and define how we want to display it in one query language. This saves a lot of time and programming overhead, as we do not have to create a separate small application to fetch, transform, and expose the data to our customers. Also, we can easily allow the user to customize the view by defining parameters he can set on a dashboard. Thanks to that, we can keep the costs for our customers low, as we use a free tool and don’t have extensive coding overhead.
If you are interested, contact Performetriks for a free EMP demo.
Resources
Dynatrace: Dynatrace | Modern cloud done right
EMP on Github: GitHub - xresch/EngineeredMonitoringPlatform
EMP Community Content: xresch EngineeredMonitoringPlatform · Discussions
EMP Youtube Channel: EMP Monitoring
EMP Setup Guide: EMP Admin - Installation And Setup
Keep up the great work! Happy Performance Engineering!!
Comments