What is Grafana? A look at its most important features for effective monitoring

What is Grafana, and what is Grafana monitoring? When Torkel Ödegaard took the decision in 2013 to fork Kibana, it heralded the beginning of the Grafana project, turning its forebear into a graph-focused and time-series dashboard tool. Ödegaard’s intention and guiding principle in creating it was to realize a dashboard that was dedicated to clarifying data by removing distractions and being a user-friendly and elegant solution. He seems to have succeeded in achieving his aims because half a million active installations don’t lie. These days you can find Grafana dashboards everywhere. Even if you’re unfamiliar with them, you may have seen one recently if you’re a fan of watching SpaceX launches!

Grafana Features

Grafana monitoring is achieved using panels. The basic building block for visualization in Grafana is the panel, and that panel can contain a graph, a Singlestat, a table, a heatmap, and freetext, and it can also integrate with both proprietary and community-created plugins too (like a clock or world map, for instance). Users are free to customize the style and format of each panel, and they can drag, drop, and resize them as they wish to create the ultimate visualization to suit their needs.

So, a Dashboard is a collection of panels, each of which holds a set of variables (things like sensor name, application, and server). These panels are arranged in a grid on the Grafana dashboard, and the user can change the data being scrutinized by switching variables, and that could be data from two different servers, for instance. Although the flexibility to customize views is one of the strongest Grafana features, users can just as easily pick up one of many ready-made dashboards to handle different data types and sources. Grafana’s large community of users and contributors has already created lots of them.

Grafana monitoring can include annotations to show particular events across the panels. Adding an annotation is achieved by putting in a custom request to Elasticsearch. Doing so makes this show up on the graph as a vertical red line. Hovering over an annotation then gives you an event description and tags (for instance) so you can track when the server returns a 5xx error code or when there’s a system restart, for instance. This makes it particularly easy to investigate system behavior and to track particular events and their consequences in an application.

Grafana Dashboard and Custom Web Apps

Grafana Dashboard and Graylog

Graylog can be used for the storage and management of web app logs and the monitoring of their performance, not just in production but also during the development stage. Grafana monitoring expresses these logs visually, to make analyzing the system more straightforward. You could legitimately describe Grafana as a web application load and performance user interface as well as a visitor flow tool. Graylog and Grafana work well together but there was no special effort made to integrate them. Graylog stores all log data in Elasticsearch, one of Grafana’s data sources, so it was easy enough to use one of the Elasticsearch indexes to connect Grafana to Graylog.

Visualizing Web Application Metrics in Grafana Dashboard

Grafana isn’t interested in error notifications or pure text logs because its main job is visualizing the data in tables, charts, and graphs. The developers created a custom Django module to track data on each web/worker request and processed response, not just reporting whether it succeeded or failed, but also providing a set of general and project-specific structured fields, including:

  • app version
  • ID for each unique request
  • response time and status
  • error code (if applicable)
  • IP address the request was sent from
  • user details (username for registered user’s email address, role, permissions)
  • device details

Django pushes these custom-structured analytical records into Graylog, which stores them in a different stream. Although Graylog dashboards can visualize this kind of data natively, they aren’t as adept at examining Grafana’s, so Grafana was adapted to visualize this analytical data. It can track application performance and load in real-time as well as retrospectively.

Top Features of Grafana

Grafana Labs is the name of the company that was created to push for the adoption of Grafana and to turn the project into a viable business. Whether you’re new to the Grafana dashboard or not, there’s a chance you may not be aware of the features that have been added to it both by Grafana Labs and its enthusiastic community.

Let’s take a look at some of the best ones:

Dashboard templating

This is one Grafana feature that’s really useful. It allows users to create a dashboard setup to suit their every need. And these templates don’t come with hardcoded values, which means that if you have a test server and a production server, the same dashboard will work with both. Templating lets you examine data at every level from the macro to the micro, so you can start with a whole country, for example, then drill down to a particular region, and keep going as far as granularity allows. These dashboards are then shareable with everyone from teams throughout your organization to the whole community.

Provisioning

It may be easy enough to set up a single dashboard with some clicking, dragging, and dropping, but some users need even more simplicity in a way that scales. So, Grafana features provisioning so you can automate setup using a script. Anything can be scripted in Grafana. For instance, when you want to create a new Kubernetes cluster, you can have Grafana automatically help with a script that already has the right server, IP address, and data sources set up and locked. This is also a way to control lots of dashboards.

Annotations

This Grafana feature lets you mark graphs, which is particularly helpful if you need to correlate data when something misbehaves. You can control-click and type on a graph to create your annotations manually, or data can be fetched from any source to populate them. (You can see an example of this in the way that Wikimedia uses annotations on its public Grafana dashboard.) A good use case would be automatically creating annotations at the time of releases. If you were to start seeing errors a little while after a new release, you could go back to your annotations and check if the errors correlate. This kind of automation is possible with the Grafana HTTP API. Lots of Grafana’s biggest customers use it for a wide range of tasks, with a common one being to set up databases and add users. This is an alternative to provisioning for automation, and there’s more you can do with it. For instance, DigitalOcean’s team used the API to include a snapshot feature that helps them to review dashboards.

Kiosk mode and playlists

Playlists are great for ‘rolling coverage’. You select which Grafana dashboards you would like to display on a monitor or TV, and it can cycle through them throughout the day. Kiosk mode lets you only show the user interface elements that you need in view-only mode. Useful tip: The Grafana Kiosk utility handles logins, switching to kiosk mode, and opening a playlist, so if a TV you want to use has no keyboard you can still set it up without hassle.

Custom plugins

You can extend Grafana’s functionality with plugins that offer extra tools, visualizations, and more. Popular examples include Worldmap Panel (which superimposes data on a map), Zabbix (which integrates with Zabbix metrics), and Influx Admin Panel (which enables database creation or lets you add users). These are just a couple of examples and there are many others besides them. Write a little code and Grafana can visualize anything that produces a timestamp. Also, Grafana Enterprise customers can access additional plugins that facilitate integrations with Datadog, New Relic, Splunk, and others.

Alerting and alert hooks

Grafana alerts can be sent through several different notifiers, including email, PagerDuty,  or Slack or texts. If these don’t work for you, it’s easy enough to code alert hooks that create different notifiers.

Teams and permissions

Where an organization has one instance of Grafana and several teams, they usually like to have the option to enforce some dashboard segregation. It used to be the case that this wasn’t possible because Grafana automatically made everyone’s dashboards accessible to everyone else. The later edition of multi-tenant mode meant that users could switch organizations but couldn’t share dashboards. Some judicious hacks could enable both, so Grafana created an easier route to achieving this. It’s now possible to create a team of users and then assign permissions on folders,  Grafana dashboards, and so on, right down to the data source level for Grafana Enterprise users.

SQL data sources

Grafana natively supports SQL, which helps you to graph any kind of data that might be held in an SQL database. High-end users are doing lots of interesting things with SQL data sources, including building business dashboards that “make sense to your boss’s boss,” (as one team put it).

Monitoring your monitoring

If you take your monitoring seriously enough to want to monitor your monitoring, Grafana features its own Prometheus HTTP endpoint that can be scraped by Prometheus, making it fairly easy to get statistics and dashboards. Once the enterprise version is up and running you’ll be able to get Google Analytics-style data access, so you can find out just how much CPU your Grafana is chewing through or how much time alerting takes.

Authentication

Grafana supports LDAP and OA and other authentication styles, and lets you map users to organizations. With Grafana Enterprise, it’s also possible to map users to teams: so if your organization uses its own authentication system, Grafana lets you map teams in your in-house systems to teams in Grafana, which automatically gives team members access to their own designated Grafana dashboards.

Grafana on Plesk

Plesk provides Grafana integration using Grafana Extension, the premier open-source software for time-series analytics. Grafana can turn all kinds of data into all kinds of visually appealing graphs and dashboards that can be customized in endless ways.

You can use Grafana to:

  • visualize data pulled from default sources, which the Plesk team can integrate for you. The default source at present is the Advanced Monitoring extension, which gathers metrics on server health.

as a more experienced Grafana user, you can pull data from any source that will integrate with Grafana. If you’d like to do this, just give the appropriate permission to the Grafana administrator.

Keeping Tabs with Plesk Obsidian’s Advanced Monitoring Extension [VIDEO]

Advanced Monitoring Extension by Plesk

Plesk is continually evolving. We released Plesk Obsidian just a few months ago, and it’s by far our most powerful management platform yet. Jam-packed with new features, Plesk Obsidian also introduces six new extensions that make the Plesk experience better than ever. Each extension offers users enhanced usability, tougher security, and maximum productivity. One of these new extensions is the Advanced Monitoring extension.

Advanced Monitoring replaces the Server Health Monitor component as the main feed for monitoring the health of your server. The Advanced Monitoring extension works in tandem with Grafana – an extension that displays server metrics as visuals, graphs and dashboards. The result is a powerful combination that optimizes both the flexibility and control of your Plesk account.

This article will fill you in on everything you need to know about the new Advanced Monitoring extension. We’ll first look at why the extension is so important and the problems it solves. Then, its main features, and finally how you can start using it.

What problems does Advanced Monitoring solve?

Keeping track of your servers can be a tricky and time-consuming task. But anyone who’s suffered an unexpected cut knows it can ultimately save you a whole lot of time and hassle. That’s why, a tool that can automate the monitoring of your servers is an invaluable asset. However, it’s important that it actually makes the process easier and upholds server monitoring best practices.

First on the best practice list is that the tool can watch various system metrics around the clock. Alerting you whenever there’s a problem. The Advanced Monitoring extension does this by tracking your server resources usage day and night.

It then notifies you when the usage of one or more resources reaches a pre-defined threshold. Because it’s essential to be instantly alerted of critical issues in order to handle them right away. Advanced Monitoring sends you emails according to your preferences and limits so you never have to worry about breaks again.

Another best practice is being able to monitor current and historical data of your server usage. This can help you get a feel for the overall state of your Plesk. Ensuring you don’t overload your server, and allowing you to plan accordingly for future needs.

Of course, all this data is useless unless it’s offered in a clear, easily-digestible manner. Grafana is the leading open source software for time series analytics. The extension displays data in Advanced Monitoring through graphs and dashboards that are highly customizable and visually appealing.

Main features of Advanced Monitoring

Let’s take an overview of Advanced Monitoring’s full feature list. With the Advanced Monitoring extension, you can:

  • See detailed reports on your server health
  • See how server health parameters change with time
  • Change the displayed time period
  • Keep track of the system resources’ usage on your server
  • Configure email notifications to inform when one or more resources’ usage reaches a pre-defined threshold (this feature is on its way).

One of the best things about the extension is that it’s ready to go out-of-the-box. Both the Advanced Monitoring and Grafana extensions are installed in Plesk Obsidian by default. You can find them located in the main left panel. The resources and services monitored by Advanced Monitoring are separated into five categories, with each one shown on its own tab.

If you upgrade to Plesk Obsidian from Plesk Onyx, Advanced Monitoring will automatically install and replace Server Health Monitor.

How to use the Advanced Monitoring extension

It’s incredibly easy to monitor your resources and server health using Advanced Monitoring. Most of its features are ready to go, but you can also customize it to suit your needs.

For example, you can select a time period for which Advanced Monitoring generates graphs. Or specify a custom period by selecting “Custom time range”. This can help you find out how your parameters change over time. As well as identify the periods when your resources usage is maximal or minimal.

You can also track your server health. Not only by looking at resources graphs, but by setting up and receiving Advanced Monitoring notifications. Advanced Monitoring can show notifications in Plesk’s interface when your resource usage exceeds its threshold and/or returns to normal value. The notifications only show for resources with notifications turned on and which have thresholds set in the Advanced Monitoring settings.

Certain resources already have their notifications turned on and their thresholds set by default. When a default threshold’s exceeded, it means a parameter is close to its critical value, which can be problematic. The automated-add on Grafana allows you to see detailed, visual reports on server health. And follow the real-time evolution for parameters that are of interest to you.

It offers numerous types of dashboards and can be used in two main ways:

  • With the data collected from default sources which were integrated for you by us.
  • (If you’re an experienced Grafana user) By harvesting data from any source you can integrate Grafana with.

We’re very excited about all Plesk Obsidian’s powerful new extensions. If you have any questions about Advanced Monitoring, be sure to leave us a comment below. Or get in touch with our friendly team of Pleskperts.

All the New Plesk Obsidian Features

Plesk’s evolution is an ongoing process since the dynamic hosting industry continues to change proactively. Cloud technologies keep revolutionizing the industry and IT professionals need help managing multiple environments and services more efficiently. Productivity, security and usability are key here and Plesk Obsidian is the timely solution. To help you better understand the level of improvements of the new Plesk version, here’s an overall view of new and updated Plesk Obsidian features. On Sep 25th 2019, Obsidian became RTM.

Advanced Monitoring & Grafana Extensions

Advanced Monitoring helps you stay in the loop regarding server and site availability. So you can get to the heart of any issues.

Since availability is key in the Obsidian release, usage is important because you want to react quick when resources become overstretched. Hence, you can now enable alerts to tell you if things like CPU, RAM, and Disk IO are becoming overburdened.

The new Grafana extension works hand in hand with Advanced Monitoring to chart info and stats in order to make them more understandable. The new Grafana charts supersede the old HealthMonitoring tool.

Moving Domains Between Subscriptions

For better security, it makes sense to separate your sites – but it’s also helpful to keep your staging and production sites apart. Now, Plesk Obsidian’s UI and CLI give anyone you choose the freedom to move domains between subscriptions. Just make sure that the target subscription has enough headroom to handle the added load, and you’re good to go.

Composer V2.0 – PHP Dependency Manager

Here are all the things you can do, or don’t need to do, with the new PHP dependency manager.

SSH access no longer needed for PHP composer

You can now run the full range of convenient PHP Composer actions, even without SSH access. It’s possible to set up environment variables, modify your composer.json and execute the Install and Update commands.

Dependencies – Just click to install and update

You can perform tests, run dependencies, or create a classmap. But what you won’t need to do is memorize all the commands and options to install them. Plesk Obsidian can handle this heavy load for you now.

Soon you won’t even need to stress about partially-updated dependencies taking down your production site. Test the latest updates until you’re sure dependencies are safe before pushing your site live.

Update dependencies when everything looks safe

Updating dependencies is a double-edged sword. It’s something you need to do to stay on top of security risks. But you also know it’s something that can break your website. Though minor updates pose less of a risk, major ones can bring the whole house of cards tumbling down.

So you’d be happy to know that you can now assess the potential risks right from the Obsidian interface. As they say – Forewarned is forearmed!

Get the correct PHP version automatically

Now your site will use the PHP version that composer.json specifies – automatically. Composer will run using the right PHP handler instead of the system default. Hence, the one the individual website uses. (There are plans to have Plesk be able to choose the appropriate PHP version and handler). Log in using SSH and type in the “php” command.

Improved File Manager UX

Now File Manager makes you even more productive thanks to mass upload and file search. You can upload and extract RAR, TAR, TAR.GZ, and TGZ archives.

Repair Kit: Self-repair Tool

Plesk Obsidian Repair Kit is a powerful self-repair tool, automatically detecting and resolving issues with Plesk or its related services. It lets you view a list of processes currently running on the server and filter them by domains and disable them. You can also see the total RAM, CPU, and Disk usage for all processes running on the server or for individual processes.

Soon it will be possible to use a list of MySQL queries and processes to discover why MySQL consumes excessive resources and which domain is to blame.

Server-side restrictions for Plesk Admins

Restricted Access mode lets you control which administrators get access to which server administration tools and settings. You’ll find it under the “Administration Tools” tab in both Power User and Service Provider views.

Control which tools, services, resources and settings your admins access while in Power User view, with Restricted Mode active. Just look under the “Hosting Tools” tab.

Essentially, you’re allowing your customers to have Plesk administrator access, without worrying about them wreaking havoc on critical and sensitive settings.

SSL It!

Make sure you keep your server protected around the clock via SSL/TLS certificates from leading, trusted certificate authorities, like Let’s Encrypt and DigiCert.

We’ll be adding support for more of these in future. In the meantime, here’s what you can use SSL It! for.

  • Buying, installing, and renewing certificates from a variety of CAs.
  • Securing your domain, webmail, the “www” subdomain, and as many aliases as you want. Get a clear view of what’s secured and what isn’t.
  • Automatically issuing, installing and renewing Let’s Encrypt certificates for domains secured with invalid certificates (self-signed, expired, or issued by a non-trusted CA.)
  • Managing HSTS, OCSP Stapling, and HTTP to HTTPS redirect.
  • Seeing the SSL/TLS status of the domain, subdomains, and domain aliases at a glance.
  • Getting an A+ rating from ssllabs.
  • Easily viewing SSL/TLS status via a small indicator on the Domain page.
  • Use the most up to date TLS versions and ciphers.

SNI for Mail Services

Keep your email conversations private, because you never know who may be spying in. You can simply secure SMTP, IMAP, and POP connections to your mail server with an SSL/TLS Certificate.

However, if you offer mail services for more than one domain, your users may have trouble with their mail clients. But you can fix this by using individual SSL/TLS certificates for each domain. Thanks to Plesk Obsidian and the SNI support provided by Postfix and MailEnable.

Personalized HTML Notifications

HTML-based email notifications help improve user experience. But they also keep you in control of your server and sites when you’re not logged into Plesk. You can read these HTML-formatted messages more easily than plain text.

Plus, the shared design makes messages more attractive and sets them apart visually from other correspondence. Use the Plesk default HTML templates or customize the look of your notifications with HTML formatting. There’s bold and italics, headers, numbered and bulleted lists, and more.

Backup Optimizations

We’ve reduced the free server disk space needed to back-up to Google, Amazon S3, FTP, and One Drive cloud storage. Now, restoring a subscription from a full server backup requires only additional free disk space equal to the subscription space. Instead of the full server backup.

Meanwhile, backing up the server to cloud storage only requires additional free disk space equal to that of two subscriptions. Rather than the entire server’s disk space. Hopefully, this has made operations faster on backups that are stored remotely. For example, you can now remove backups stored in the cloud four times quicker than before.

Revised Plesk UI and More Key Improvements

We’ve modernized the overall Plesk UI to address some of the UX shortfalls our users reported. Notice that the Plesk interface now looks more like plesk.com. There’s more to come – but for now we’ve improved the appearance and behavior of individual Plesk elements, colors and font sizes. Then, aligned all elements to a grid.

There have also been some key additions: 

  1. Precompiled PageSpeed:
    The PageSpeed module is now precompiled with NGINX.
  2. Automatic Restarts:
    Plesk services that crash on systemd-based Linux OSes (CentOS7, RHEL 7, Ubuntu 16.04/18.04, Debian 8/9) will now restart automatically. Apache graceful restart is now strong enough to be set as the default to reduce downtime for websites.
  3. Immediate Default Security:
    Secured by default: mod_security and fail2ban v0.10 are active straight away.
  4. Easier Plesk Access:
    Plesk entry point has been optimized for Clouds like Amazon and Digital Ocean. Access Plesk via https://<host-name-or-IP> without specifying the TCP port 8443.
  5. Default HTTP to HTTPS:
    Newly created websites have an SEO-friendly HTTP to HTTPS redirect enabled by default.
  6. PHP-FPM & Apache Tweaks:
    Services improvements will give you more reliability with restarts. PHP engines supplied with Plesk contain popular PHP extensions like sodium, exif, and fileinfo.

It is worth mentioning that, starting with Plesk Obsidian, Plesk is announcing a monthly release cycle in order to always deliver value through secure and stable software to our Partners and Customers. Once you upgraded to Plesk Obsidian and allowed automatic updates through Plesk Server settings, you will always and automatically be on the latest version of the software. See below a complete comparison between Onyx and Obsidian – the complete list of arguments about why to upgrade:

Plesk Onyx vs Plesk Obsidian

Plesk Onyx vs Plesk Obsidian INFOGRAPHIC

Check more podcasts