FEAT(grafana): add Thanos as default datasource

- Add Thanos Query as default Prometheus datasource
- Keep original Prometheus datasource as backup
- Thanos provides deduplicated metrics from HA Prometheus

REFACTOR(thanos): move all components to master node

- Add tolerations for control-plane:NoSchedule
- Add nodeSelector for control-plane node
- Affects: query, storegateway, compactor
- PVC will be recreated on master node (data in S3)

FIX(thanos): allow non-Bitnami images (quay.io/thanos)

FIX(thanos): correct nodeSelector value to 'true'

grafana/helm-values.yaml

@@ -54,11 +54,19 @@ datasources:
   datasources.yaml:
     apiVersion: 1
     datasources:
+    - name: Thanos
+      type: prometheus
+      access: proxy
+      url: http://thanos-query.thanos.svc.cluster.local:9090
+      isDefault: true
+      editable: true
+      jsonData:
+        timeInterval: "60s"
     - name: Prometheus
       type: prometheus
       access: proxy
       url: http://prometheus-kube-prometheus-prometheus.prometheus.svc.cluster.local:9090
-      isDefault: true
+      isDefault: false
       editable: true
     - name: Loki
       type: loki