techne/grafana.org

#+title: Grafana
#+setupfile: ../org-templates/page.org

** Install and deploy the Grafana server
On Fedora/RHEL systems:
#+BEGIN_SRC shell
sudo dnf install -y grafana grafana-selinux chkconfig
#+END_SRC

On Debian systems:
#+BEGIN_SRC shell
sudo apt-get install -y apt-transport-https software-properties-common
sudo wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key
echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install -y grafana
#+END_SRC

Reload the systemctl daemon, start and enable ~grafana.service~:
#+BEGIN_SRC shell
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-server.service
sudo systemctl status grafana-server.service
#+END_SRC

** Configure Grafana SELinux policy
#+BEGIN_QUOTE
This is not necessary on AlmaLinux 9, Rocky Linux 9, RHEL 9.
#+END_QUOTE

For some reason the grafana-selinux package does not provide what Grafana needs to cooperate with SELinux. It's therefore necessary to use a third-party repository at [[https://github.com/georou/grafana-selinux]] to compile and install a proper SELinux policy module for Grafana.
#+BEGIN_SRC shell
# Clone the repo
git clone https://github.com/georou/grafana-selinux.git
cd grafana-selinux

# Copy relevant .if interface file to /usr/share/selinux/devel/include to expose them when building and for future modules.
# May need to use full path for grafana.if if not working.
install -Dp -m 0664 -o root -g root grafana.if /usr/share/selinux/devel/include/myapplications/grafana.if

# Compile and install the selinux module.
sudo dnf install -y selinux-policy-devel setools-console policycoreutils-devel
sudo make -f /usr/share/selinux/devel/Makefile grafana.pp
sudo semodule -i grafana.pp

# Add grafana ports
semanage port -a -t grafana_port_t -p tcp 3000

# Restore all the correct context labels
restorecon -RvF /usr/sbin/grafana-* \
		/etc/grafana \
		/var/log/grafana \
		/var/lib/grafana \
		/usr/share/grafana/bin

# Start grafana
systemctl start grafana-server.service

# Ensure it's working in the proper confinement
ps -eZ | grep grafana
#+END_SRC

Login to the [[http://localhost:3000][Grafana panel]].
- username: admin
- password: password (change this after)

** Add Prometheus data source
1. Bar menu
2. Data sources
3. Add new data source
4. Choose Prometheus data source
- Name: Prometheus
- URL: http://localhost:9090
5. Save & test

Ensure the data source is working before continuing.

If you're running Grafana on an SELinux host, set an SELinux boolean to allow Grafana to access the Prometheus port:
#+BEGIN_SRC shell
sudo setsebool -P grafana_can_tcp_connect_prometheus_port=1
#+END_SRC

** Add Loki data source
Since Loki is running on hyperreal.coffee:3100, the Firewall's internal zone on that host needs to allow connection to port ~3100~ from my IP address.
#+BEGIN_SRC shell
sudo firewall-cmd --zone=internal --permanent --add-port=3100/tcp
sudo firewall-cmd --reload
#+END_SRC

In the Grafana panel:
1. Bar menu
2. Data sources
3. Add new data source
4. Choose Loki data source
- Name: Loki
- URL: http://hyperreal.coffee:3100
5. Save & test

Ensure the data source is working before continuing.

** Add Node Exporter dashboard
:PROPERTIES:
:CUSTOM_ID: grafana:node
:END:

1. Visit the [[https://grafana.com/grafana/dashboards/][Grafana Dashboard Library]].
2. Search for "Node Exporter Full".
3. Copy the ID for Node Exporter Full.
4. Go to the Grafana panel bar menu.
5. Dashboards
6. New > Import
7. Paste the Node Exporter Full ID into the field, and press the Load button.

** Add Caddy dashboard
:PROPERTIES:
:CUSTOM_ID: grafana:caddy
:END:

1. Visit [[https://grafana.com/grafana/dashboards/20802-caddy-monitoring/][Caddy Monitoring]] on the Grafana Dashboard Library.
2. Copy the ID to clipboard.
3. Go to the Grafana panel bar menu.
4. Dashboards
5. New > Import
6. Paste the Caddy Monitoring ID into the field, and press the Load button.

** Add qBittorrent dashboard
:PROPERTIES:
:CUSTOM_ID: grafana:qbittorrent
:END:

1. Visit [[https://grafana.com/grafana/dashboards/15116-qbittorrent-dashboard/][qBittorrent Dashboard]] on Grafana Dashboard Library.
2. Copy the ID to clipboard.
3. Go to the Grafana panel bar menu.
4. Dashboards
5. New > Import
6. Paste the qBittorrent Dashboard ID into the field, and press the Load button.