refactor: build event tags

README.md

@@ -3,7 +3,7 @@
     <img src="doc/img/Nightingale_L_V.png" alt="nightingale - cloud native monitoring" width="100" /></a>
 </p>
 <p align="center">
-  <b>开源告警管理专家</b>
+  <b>开源告警管理专家 一体化的可观测平台</b>
 </p>
 
 <p align="center">
@@ -27,86 +27,77 @@
 
 [English](./README_en.md) | [中文](./README.md)
 
-## 夜莺是什么
+## 夜莺 Nightingale 是什么
 
-夜莺监控（Nightingale）是一款侧重告警的监控类开源项目。类似 Grafana 的数据源集成方式，夜莺也是对接多种既有的数据源，不过 Grafana 侧重在可视化，夜莺是侧重在告警引擎、告警事件的处理和分发。
+> 夜莺 Nightingale 是什么，解决什么问题？以大家都很熟悉的 Grafana 做个类比，Grafana 擅长对接各种各样的数据源，然后提供灵活、强大、好看的可视化面板。夜莺则擅长对接各种多样的数据源，提供灵活、强大、高效的监控告警管理能力。从发展路径和定位来说，夜莺和 Grafana 很像，可以总结为一句话：可视化就用 Grafana，监控告警就找夜莺。
+>
+> 在可视化领域，Grafana 是毫无争议的领导者，Grafana 在影响力、装机量、用户群、开发者数量等各个维度的数字上，相比夜莺都是追赶的榜样。巨无霸往往都是从一个切入点打开局面的，Grafana Labs 有了在可视化领域 Grafana 这个王牌，逐步扩展到整个可观测性方向，比如 Logging 维度有 Loki，Tracing 维度有 Tempo，Profiling 维度有收购来的 Pyroscope，On-call 维度有同样是收购来的 Grafana-OnCall 项目，还有时序数据库 Mimir、eBPF 采集器 Beyla、OpenTelemetry 采集器 Alloy、前端监控 SDK Faro，最终构成了一个完整的可观测性工具矩阵，但整个飞轮都是从 Grafana 项目开始转动起来的。
+>
+>夜莺，则是从监控告警这个切入点打开局面，也逐步横向做了相应扩展，比如夜莺也自研了可视化面板，如果你想有一个 all-in-one 的监控告警+可视化的工具，那么用夜莺也是正确的选择；比如 OnCall 方向，夜莺可以和 [Flashduty SaaS](https://flashcat.cloud/product/flashcat-duty/) 服务无缝的集成；在采集器方向，夜莺有配套的 [Categraf](https://flashcat.cloud/product/categraf)，可以一个采集器中管理所有的 exporter，并同时支持指标和日志的采集，极大减轻工程师维护的采集器数量和工作量（这个点太痛了，你可能也遇到过业务团队吐槽采集器数量比业务应用进程数量还多的窘况吧）。
 
-夜莺监控项目，最初由滴滴开发和开源，并于 2022 年 5 月 11 日，捐赠予中国计算机学会开源发展委员会（CCF ODC），为 CCF ODC 成立后接受捐赠的第一个开源项目。
+夜莺 Nightingale 作为一款开源云原生监控工具，最初由滴滴开发和开源，并于 2022 年 5 月 11 日，捐赠予中国计算机学会开源发展委员会（CCF ODC），为 CCF ODC 成立后接受捐赠的第一个开源项目。在 GitHub 上有超过 10000 颗星，是广受关注和使用的开源监控工具。夜莺的核心研发团队，也是 Open-Falcon 项目原核心研发人员，从 2014 年（Open-Falcon 是 2014 年开源）算起来，也有 10 年了，只为把监控做到极致。
 
-## 夜莺的工作逻辑
 
-很多用户已经自行采集了指标、日志数据，此时就把存储库（VictoriaMetrics、ElasticSearch等）作为数据源接入夜莺，即可在夜莺里配置告警规则、通知规则，完成告警事件的生成和派发。
+## 快速开始
+- 👉 [文档中心](https://flashcat.cloud/docs/) | [下载中心](https://flashcat.cloud/download/nightingale/)
+- ❤️ [报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml)
+- ℹ️ 为了提供更快速的访问体验，上述文档和下载站点托管于 [FlashcatCloud](https://flashcat.cloud)
+- 💡 前后端代码分离，前端代码仓库：[https://github.com/n9e/fe](https://github.com/n9e/fe)
 
-![夜莺产品架构](doc/img/readme/20240221152601.png)
+## 功能特点
 
-夜莺项目本身不提供监控数据采集能力。推荐您使用 [Categraf](https://github.com/flashcatcloud/categraf) 作为采集器，可以和夜莺丝滑对接。
+- 对接多种时序库，实现统一监控告警管理：支持对接的时序库包括 Prometheus、VictoriaMetrics、Thanos、Mimir、M3DB、TDengine 等。
+- 对接日志库，实现针对日志的监控告警：支持对接的日志库包括 ElasticSearch、Loki 等。
+- 专业告警能力：内置支持多种告警规则，可以扩展支持常见通知媒介，支持告警屏蔽/抑制/订阅/自愈、告警事件管理。
+- 高性能可视化引擎：支持多种图表样式，内置众多 Dashboard 模版，也可导入 Grafana 模版，开箱即用，开源协议商业友好。
+- 支持常见采集器：支持 [Categraf](https://flashcat.cloud/product/categraf)、Telegraf、Grafana-agent、Datadog-agent、各种 Exporter 作为采集器，没有什么数据是不能监控的。
+- 👀无缝搭配 [Flashduty](https://flashcat.cloud/product/flashcat-duty/)：实现告警聚合收敛、认领、升级、排班、IM集成，确保告警处理不遗漏，减少打扰，高效协同。
 
-[Categraf](https://github.com/flashcatcloud/categraf) 可以采集操作系统、网络设备、各类中间件、数据库的监控数据，通过 Remote Write 协议推送给夜莺，夜莺把监控数据转存到时序库（如 Prometheus、VictoriaMetrics 等），并提供告警和可视化能力。
 
-对于个别边缘机房，如果和中心夜莺服务端网络链路不好，希望提升告警可用性，夜莺也提供边缘机房告警引擎下沉部署模式，这个模式下，即便边缘和中心端网络割裂，告警功能也不受影响。
+## 截图演示
+
+
+你可以在页面的右上角，切换语言和主题，目前我们支持英语、简体中文、繁体中文。
+
+![语言切换](doc/img/readme/n9e-switch-i18n.png)
+
+即时查询，类似 Prometheus 内置的查询分析页面，做 ad-hoc 查询，夜莺做了一些 UI 优化，同时提供了一些内置 promql 指标，让不太了解 promql 的用户也可以快速查询。
+
+![即时查询](doc/img/readme/20240513103305.png)
+
+当然，也可以直接通过指标视图查看，有了指标视图，即时查询基本可以不用了，或者只有高端玩家使用即时查询，普通用户直接通过指标视图查询即可。
+
+![指标视图](doc/img/readme/20240513103530.png)
+
+夜莺内置了常用仪表盘，可以直接导入使用。也可以导入 Grafana 仪表盘，不过只能兼容 Grafana 基本图表，如果已经习惯了 Grafana 建议继续使用 Grafana 看图，把夜莺作为一个告警引擎使用。
+
+![内置仪表盘](doc/img/readme/20240513103628.png)
+
+除了内置的仪表盘，也内置了很多告警规则，开箱即用。
+
+![内置告警规则](doc/img/readme/20240513103825.png)
+
+
+
+## 产品架构
+
+社区使用夜莺最多的场景就是使用夜莺做告警引擎，对接多套时序库，统一告警规则管理。绘图仍然使用 Grafana 居多。作为一个告警引擎，夜莺的产品架构如下：
+
+![产品架构](doc/img/readme/20240221152601.png)
+
+对于个别边缘机房，如果和中心夜莺服务端网络链路不好，希望提升告警可用性，我们也提供边缘机房告警引擎下沉部署模式，这个模式下，即便网络割裂，告警功能也不受影响。
 
 ![边缘部署模式](doc/img/readme/20240222102119.png)
 
-> 上图中，机房A和中心机房的网络链路很好，所以直接由中心端的夜莺进程做告警引擎，机房B和中心机房的网络链路不好，所以在机房B部署了 `n9e-edge` 做告警引擎，对机房B的数据源做告警判定。
 
-## 告警降噪、升级、协同
-
-夜莺的侧重点是做告警引擎，即负责产生告警事件，并根据规则做灵活派发，内置支持 20 种通知媒介（电话、短信、邮件、钉钉、飞书、企微、Slack 等）。
-
-如果您有更高级的需求，比如：
-
-- 想要把公司的多套监控系统产生的事件聚拢到一个平台，统一做收敛降噪、响应处理、数据分析
-- 想要支持人员的排班，践行 On-call 文化，想要支持告警认领、升级（避免遗漏）、协同处理
-
-那夜莺是不合适的，推荐您选用 [FlashDuty](https://flashcat.cloud/product/flashcat-duty/) 这样的 On-call 产品，产品简单易用，也有免费套餐。
-
-
-## 相关资料 & 交流渠道
-- 📚 [夜莺介绍PPT](https://mp.weixin.qq.com/s/Mkwx_46xrltSq8NLqAIYow) 对您了解夜莺各项关键特性会有帮助（PPT链接在文末）
-- 👉 [文档中心](https://flashcat.cloud/docs/) 为了更快的访问速度，站点托管在 [FlashcatCloud](https://flashcat.cloud)
-- ❤️ [报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml) 写清楚问题描述、复现步骤、截图等信息，更容易得到答案
-- 💡 前后端代码分离，前端代码仓库：[https://github.com/n9e/fe](https://github.com/n9e/fe)
-- 🎯 关注[这个公众号](https://gitlink.org.cn/UlricQin)了解更多夜莺动态和知识
-- 🌟 加我微信：`picobyte`（我已关闭好友验证）拉入微信群，备注：`夜莺互助群`，如果已经把夜莺上到生产环境，可联系我拉入资深监控用户群
-
-
-## 关键特性简介
-
-![夜莺告警规则](doc/img/readme/2025-05-23_18-43-37.png)
-
-- 夜莺支持告警规则、屏蔽规则、订阅规则、通知规则，内置支持 20 种通知媒介，支持消息模板自定义
-- 支持事件管道，对告警事件做 Pipeline 处理，方便和自有系统做自动化整合，比如给告警事件附加一些元信息，对事件做 relabel
-- 支持业务组概念，引入权限体系，分门别类管理各类规则
-- 很多数据库、中间件内置了告警规则，可以直接导入使用，也可以直接导入 Prometheus 的告警规则
-- 支持告警自愈，即告警之后自动触发一个脚本执行一些预定义的逻辑，比如清理一下磁盘、抓一下现场等
-
-![夜莺事件大盘](doc/img/readme/2025-05-30_08-49-28.png)
-
-- 夜莺存档了历史告警事件，支持多维度的查询和统计
-- 支持灵活的聚合分组，一目了然看到公司的告警事件分布情况
-
-![夜莺集成中心](doc/img/readme/2025-05-23_18-46-06.png)
-
-- 夜莺内置常用操作系统、中间件、数据库的的指标说明、仪表盘、告警规则，不过都是社区贡献的，整体也是参差不齐
-- 夜莺直接接收 Remote Write、OpenTSDB、Datadog、Falcon 等多种协议的数据，故而可以和各类 Agent 对接
-- 夜莺支持 Prometheus、ElasticSearch、Loki、TDEngine 等多种数据源，可以对其中的数据做告警
-- 夜莺可以很方便内嵌企业内部系统，比如 Grafana、CMDB 等，甚至可以配置这些内嵌系统的菜单可见性
-
-
-![夜莺仪表盘](doc/img/readme/2025-05-23_18-49-02.png)
-
-- 夜莺支持仪表盘功能，支持常见的图表类型，也内置了一些仪表盘，上图是其中一个仪表盘的截图。
-- 如果你已经习惯了 Grafana，建议仍然使用 Grafana 看图。Grafana 在看图方面道行更深。
-- 机器相关的监控数据，如果是 Categraf 采集的，建议使用夜莺自带的仪表盘查看，因为 Categraf 的指标命名 Follow 的是 Telegraf 的命名方式，和 Node Exporter 不同
-- 因为夜莺有个业务组的概念，机器可以归属不同的业务组，有时在仪表盘里只想查看当前所属业务组的机器，所以夜莺的仪表盘可以和业务组联动
+## 交流渠道
+- 报告Bug，优先推荐提交[夜莺GitHub Issue](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=kind%2Fbug&projects=&template=bug_report.yml)
+- 关注[这个公众号](https://gitlink.org.cn/UlricQin)了解更多夜莺动态和知识
+- 加我微信：`picobyte`（我已关闭好友验证）拉入微信群，备注：`夜莺互助群`
 
 ## 广受关注
 [![Stargazers over time](https://api.star-history.com/svg?repos=ccfos/nightingale&type=Date)](https://star-history.com/#ccfos/nightingale&Date)
 
-## 感谢众多企业的信赖
-
-![夜莺客户](doc/img/readme/logos.png)
-
 ## 社区共建
 - ❇️ 请阅读浏览[夜莺开源项目和社区治理架构草案](./doc/community-governance.md)，真诚欢迎每一位用户、开发者、公司以及组织，使用夜莺监控、积极反馈 Bug、提交功能需求、分享最佳实践，共建专业、活跃的夜莺开源社区。
 - ❤️ 夜莺贡献者

alert/astats/stats.go

@@ -25,7 +25,6 @@ type Stats struct {
 	CounterHeartbeatErrorTotal  *prometheus.CounterVec
 	CounterSubEventTotal        *prometheus.CounterVec
 	GaugeQuerySeriesCount       *prometheus.GaugeVec
-	GaugeRuleEvalDuration       *prometheus.GaugeVec
 	GaugeNotifyRecordQueueSize  prometheus.Gauge
 }
 
@@ -137,13 +136,6 @@ func NewSyncStats() *Stats {
 		Help:      "The size of notify record queue.",
 	})
 
-	GaugeRuleEvalDuration := prometheus.NewGaugeVec(prometheus.GaugeOpts{
-		Namespace: namespace,
-		Subsystem: subsystem,
-		Name:      "rule_eval_duration_ms",
-		Help:      "Duration of rule eval in milliseconds.",
-	}, []string{"rule_id", "datasource_id"})
-
 	CounterVarFillingQuery := prometheus.NewCounterVec(prometheus.CounterOpts{
 		Namespace: namespace,
 		Subsystem: subsystem,
@@ -166,7 +158,6 @@ func NewSyncStats() *Stats {
 		CounterHeartbeatErrorTotal,
 		CounterSubEventTotal,
 		GaugeQuerySeriesCount,
-		GaugeRuleEvalDuration,
 		GaugeNotifyRecordQueueSize,
 		CounterVarFillingQuery,
 	)
@@ -186,7 +177,6 @@ func NewSyncStats() *Stats {
 		CounterHeartbeatErrorTotal:  CounterHeartbeatErrorTotal,
 		CounterSubEventTotal:        CounterSubEventTotal,
 		GaugeQuerySeriesCount:       GaugeQuerySeriesCount,
-		GaugeRuleEvalDuration:       GaugeRuleEvalDuration,
 		GaugeNotifyRecordQueueSize:  GaugeNotifyRecordQueueSize,
 		CounterVarFillingQuery:      CounterVarFillingQuery,
 	}

alert/dispatch/dispatch.go

@@ -16,6 +16,7 @@ import (
 	"github.com/ccfos/nightingale/v6/alert/astats"
 	"github.com/ccfos/nightingale/v6/alert/common"
 	"github.com/ccfos/nightingale/v6/alert/pipeline"
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/relabel"
 	"github.com/ccfos/nightingale/v6/alert/sender"
 	"github.com/ccfos/nightingale/v6/memsto"
 	"github.com/ccfos/nightingale/v6/models"
@@ -81,7 +82,8 @@ func NewDispatch(alertRuleCache *memsto.AlertRuleCacheType, userCache *memsto.Us
 		Astats: astats,
 	}
 
-	pipeline.Init()
+	relabel.Init()
+
 	return notify
 }
 
@@ -148,8 +150,8 @@ func (e *Dispatch) reloadTpls() error {
 
 func (e *Dispatch) HandleEventWithNotifyRule(eventOrigin *models.AlertCurEvent) {
 
-	if len(eventOrigin.NotifyRuleIds) > 0 {
-		for _, notifyRuleId := range eventOrigin.NotifyRuleIds {
+	if len(eventOrigin.NotifyRuleIDs) > 0 {
+		for _, notifyRuleId := range eventOrigin.NotifyRuleIDs {
 			// 深拷贝新的 event，避免并发修改 event 冲突
 			eventCopy := eventOrigin.DeepCopy()
 
@@ -163,7 +165,7 @@ func (e *Dispatch) HandleEventWithNotifyRule(eventOrigin *models.AlertCurEvent)
 				continue
 			}
 
-			var processors []models.Processor
+			var processors []pipeline.Processor
 			for _, pipelineConfig := range notifyRule.PipelineConfigs {
 				if !pipelineConfig.Enable {
 					continue
@@ -180,24 +182,26 @@ func (e *Dispatch) HandleEventWithNotifyRule(eventOrigin *models.AlertCurEvent)
 					continue
 				}
 
-				processors = append(processors, e.eventProcessorCache.GetProcessorsById(pipelineConfig.PipelineId)...)
+				for _, p := range eventPipeline.Processors {
+					processor, err := pipeline.GetProcessorByType(p.Typ, p.Config)
+					if err != nil {
+						logger.Warningf("notify_id: %d, event:%+v, processor:%+v type not found", notifyRuleId, eventCopy, p)
+						continue
+					}
+					processors = append(processors, processor)
+				}
 			}
 
 			for _, processor := range processors {
 				logger.Infof("before processor notify_id: %d, event:%+v, processor:%+v", notifyRuleId, eventCopy, processor)
-				eventCopy, res, err := processor.Process(e.ctx, eventCopy)
-				logger.Infof("after processor notify_id: %d, event:%+v, processor:%+v, res:%v, err:%v", notifyRuleId, eventCopy, processor, res, err)
+				processor.Process(e.ctx, eventCopy)
+				logger.Infof("after processor notify_id: %d, event:%+v, processor:%+v", notifyRuleId, eventCopy, processor)
 				if eventCopy == nil {
 					logger.Warningf("notify_id: %d, event:%+v, processor:%+v, event is nil", notifyRuleId, eventCopy, processor)
 					break
 				}
 			}
 
-			if eventCopy == nil {
-				// 如果 eventCopy 为 nil，说明 eventCopy 被 processor drop 掉了, 不再发送通知
-				continue
-			}
-
 			// notify
 			for i := range notifyRule.NotifyConfigs {
 				if !NotifyRuleApplicable(&notifyRule.NotifyConfigs[i], eventCopy) {
@@ -503,6 +507,11 @@ func (e *Dispatch) HandleEventNotify(event *models.AlertCurEvent, isSubscribe bo
 		return
 	}
 
+	if e.blockEventNotify(rule, event) {
+		logger.Infof("block event notify: rule_id:%d event:%+v", rule.Id, event)
+		return
+	}
+
 	fillUsers(event, e.userCache, e.userGroupCache)
 
 	var (
@@ -539,6 +548,25 @@ func (e *Dispatch) HandleEventNotify(event *models.AlertCurEvent, isSubscribe bo
 	}
 }
 
+func (e *Dispatch) blockEventNotify(rule *models.AlertRule, event *models.AlertCurEvent) bool {
+	ruleType := rule.GetRuleType()
+
+	// 若为机器则先看机器是否删除
+	if ruleType == models.HOST {
+		host, ok := e.targetCache.Get(event.TagsMap["ident"])
+		if !ok || host == nil {
+			return true
+		}
+	}
+
+	// 恢复通知，检测规则配置是否改变
+	// if event.IsRecovered && event.RuleHash != rule.Hash() {
+	// 	return true
+	// }
+
+	return false
+}
+
 func (e *Dispatch) handleSubs(event *models.AlertCurEvent) {
 	// handle alert subscribes
 	subscribes := make([]*models.AlertSubscribe, 0)

alert/dispatch/log.go

@@ -24,7 +24,7 @@ func LogEvent(event *models.AlertCurEvent, location string, err ...error) {
 		location,
 		event.RuleId,
 		event.SubRuleId,
-		event.NotifyRuleIds,
+		event.NotifyRuleIDs,
 		event.Cluster,
 		event.TagsJSON,
 		event.TriggerValue,

alert/eval/alert_rule.go

@@ -93,7 +93,7 @@ func (s *Scheduler) syncAlertRules() {
 		}
 
 		ruleType := rule.GetRuleType()
-		if rule.IsPrometheusRule() || rule.IsInnerRule() {
+		if rule.IsPrometheusRule() || rule.IsLokiRule() || rule.IsTdengineRule() || rule.IsClickHouseRule() || rule.IsElasticSearch() {
 			datasourceIds := s.datasourceCache.GetIDsByDsCateAndQueries(rule.Cate, rule.DatasourceQueries)
 			for _, dsId := range datasourceIds {
 				if !naming.DatasourceHashRing.IsHit(strconv.FormatInt(dsId, 10), fmt.Sprintf("%d", rule.Id), s.aconf.Heartbeat.Endpoint) {

alert/eval/eval.go

@@ -172,7 +172,7 @@ func (arw *AlertRuleWorker) Eval() {
 	case models.LOKI:
 		anomalyPoints, err = arw.GetPromAnomalyPoint(cachedRule.RuleConfig)
 	default:
-		anomalyPoints, recoverPoints, err = arw.GetAnomalyPoint(cachedRule, arw.Processor.DatasourceId())
+		anomalyPoints, recoverPoints = arw.GetAnomalyPoint(cachedRule, arw.Processor.DatasourceId())
 	}
 
 	if err != nil {
@@ -232,10 +232,6 @@ func (arw *AlertRuleWorker) Stop() {
 
 func (arw *AlertRuleWorker) GetPromAnomalyPoint(ruleConfig string) ([]models.AnomalyPoint, error) {
 	var lst []models.AnomalyPoint
-	start := time.Now()
-	defer func() {
-		arw.Processor.Stats.GaugeRuleEvalDuration.WithLabelValues(fmt.Sprintf("%v", arw.Rule.Id), fmt.Sprintf("%v", arw.Processor.DatasourceId())).Set(float64(time.Since(start).Milliseconds()))
-	}()
 
 	var rule *models.PromRuleConfig
 	if err := json.Unmarshal([]byte(ruleConfig), &rule); err != nil {
@@ -746,10 +742,6 @@ func combine(paramKeys []string, paraMap map[string][]string, index int, current
 
 func (arw *AlertRuleWorker) GetHostAnomalyPoint(ruleConfig string) ([]models.AnomalyPoint, error) {
 	var lst []models.AnomalyPoint
-	start := time.Now()
-	defer func() {
-		arw.Processor.Stats.GaugeRuleEvalDuration.WithLabelValues(fmt.Sprintf("%v", arw.Rule.Id), fmt.Sprintf("%v", arw.Processor.DatasourceId())).Set(float64(time.Since(start).Milliseconds()))
-	}()
 
 	var rule *models.HostRuleConfig
 	if err := json.Unmarshal([]byte(ruleConfig), &rule); err != nil {
@@ -1422,18 +1414,13 @@ func fillVar(curRealQuery string, paramKey string, val string) string {
 	return curRealQuery
 }
 
-func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64) ([]models.AnomalyPoint, []models.AnomalyPoint, error) {
+func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64) ([]models.AnomalyPoint, []models.AnomalyPoint) {
 	// 获取查询和规则判断条件
-	start := time.Now()
-	defer func() {
-		arw.Processor.Stats.GaugeRuleEvalDuration.WithLabelValues(fmt.Sprintf("%v", arw.Rule.Id), fmt.Sprintf("%v", arw.Processor.DatasourceId())).Set(float64(time.Since(start).Milliseconds()))
-	}()
-
 	points := []models.AnomalyPoint{}
 	recoverPoints := []models.AnomalyPoint{}
 	ruleConfig := strings.TrimSpace(rule.RuleConfig)
 	if ruleConfig == "" {
-		logger.Warningf("rule_eval:%d ruleConfig is blank", rule.Id)
+		logger.Warningf("rule_eval:%d promql is blank", rule.Id)
 		arw.Processor.Stats.CounterRuleEvalErrorTotal.WithLabelValues(fmt.Sprintf("%v", arw.Processor.DatasourceId()), GET_RULE_CONFIG, arw.Processor.BusiGroupCache.GetNameByBusiGroupId(arw.Rule.GroupId), fmt.Sprintf("%v", arw.Rule.Id)).Inc()
 		arw.Processor.Stats.GaugeQuerySeriesCount.WithLabelValues(
 			fmt.Sprintf("%v", arw.Rule.Id),
@@ -1441,7 +1428,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 			"",
 		).Set(0)
 
-		return points, recoverPoints, fmt.Errorf("rule_eval:%d ruleConfig is blank", rule.Id)
+		return points, recoverPoints
 	}
 
 	var ruleQuery models.RuleQuery
@@ -1449,7 +1436,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 	if err != nil {
 		logger.Warningf("rule_eval:%d promql parse error:%s", rule.Id, err.Error())
 		arw.Processor.Stats.CounterRuleEvalErrorTotal.WithLabelValues(fmt.Sprintf("%v", arw.Processor.DatasourceId()), GET_RULE_CONFIG, arw.Processor.BusiGroupCache.GetNameByBusiGroupId(arw.Rule.GroupId), fmt.Sprintf("%v", arw.Rule.Id)).Inc()
-		return points, recoverPoints, fmt.Errorf("rule_eval:%d promql parse error:%s", rule.Id, err.Error())
+		return points, recoverPoints
 	}
 
 	arw.Inhibit = ruleQuery.Inhibit
@@ -1469,8 +1456,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 					fmt.Sprintf("%v", arw.Processor.DatasourceId()),
 					fmt.Sprintf("%v", i),
 				).Set(-2)
-
-				return points, recoverPoints, fmt.Errorf("rule_eval:%d datasource:%d not exists", rule.Id, dsId)
+				continue
 			}
 
 			ctx := context.WithValue(context.Background(), "delay", int64(rule.Delay))
@@ -1485,7 +1471,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 					fmt.Sprintf("%v", i),
 				).Set(-1)
 
-				return points, recoverPoints, fmt.Errorf("rule_eval:%d query data error: %v", rule.Id, err)
+				continue
 			}
 
 			arw.Processor.Stats.GaugeQuerySeriesCount.WithLabelValues(
@@ -1519,7 +1505,6 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 		for _, query := range ruleQuery.Queries {
 			ref, unit, err := GetQueryRefAndUnit(query)
 			if err != nil {
-				logger.Warningf("rule_eval rid:%d query:%+v get ref and unit error:%s", rule.Id, query, err.Error())
 				continue
 			}
 			unitMap[ref] = unit
@@ -1686,5 +1671,5 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 		}
 	}
 
-	return points, recoverPoints, nil
+	return points, recoverPoints
 }

alert/mute/mute.go

@@ -44,12 +44,6 @@ func TimeSpanMuteStrategy(rule *models.AlertRule, event *models.AlertCurEvent) b
 	triggerTime := tm.Format("15:04")
 	triggerWeek := strconv.Itoa(int(tm.Weekday()))
 
-	if rule.EnableDaysOfWeek == "" {
-		// 如果规则没有配置生效时间，则默认全天生效
-
-		return false
-	}
-
 	enableStime := strings.Fields(rule.EnableStime)
 	enableEtime := strings.Fields(rule.EnableEtime)
 	enableDaysOfWeek := strings.Split(rule.EnableDaysOfWeek, ";")
@@ -135,7 +129,7 @@ func EventMuteStrategy(event *models.AlertCurEvent, alertMuteCache *memsto.Alert
 	}
 
 	for i := 0; i < len(mutes); i++ {
-		if MatchMute(event, mutes[i]) {
+		if matchMute(event, mutes[i]) {
 			return true, mutes[i].Id
 		}
 	}
@@ -143,12 +137,13 @@ func EventMuteStrategy(event *models.AlertCurEvent, alertMuteCache *memsto.Alert
 	return false, 0
 }
 
-// MatchMute 如果传入了clock这个可选参数，就表示使用这个clock表示的时间，否则就从event的字段中取TriggerTime
-func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int64) bool {
+// matchMute 如果传入了clock这个可选参数，就表示使用这个clock表示的时间，否则就从event的字段中取TriggerTime
+func matchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int64) bool {
 	if mute.Disabled == 1 {
 		return false
 	}
 
+
 	// 如果不是全局的，判断 匹配的 datasource id
 	if len(mute.DatasourceIdsJson) != 0 && mute.DatasourceIdsJson[0] != 0 && event.DatasourceId != 0 {
 		idm := make(map[int64]struct{}, len(mute.DatasourceIdsJson))
@@ -171,7 +166,7 @@ func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int
 		if len(clock) > 0 {
 			ts = clock[0]
 		}
-
+		
 		if !mute.IsWithinPeriodicMute(ts) {
 			return false
 		}

alert/pipeline/pipeline.go

@@ -1,12 +1,12 @@
 package pipeline
 
 import (
-	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/aisummary"
-	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
-	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/eventdrop"
-	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/eventupdate"
-	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/relabel"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
 )
 
-func Init() {
+func Pipeline(ctx *ctx.Context, event *models.AlertCurEvent, processors []Processor) {
+	for _, processor := range processors {
+		processor.Process(ctx, event)
+	}
 }

alert/pipeline/processor.go

@@ -1,25 +1,33 @@
-package models
+package pipeline
 
 import (
 	"fmt"
 	"strings"
 
+	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/ctx"
 )
 
+// Processor 是处理器接口，所有处理器类型都需要实现此接口
 type Processor interface {
-	Init(settings interface{}) (Processor, error) // 初始化配置
-	Process(ctx *ctx.Context, event *AlertCurEvent) (*AlertCurEvent, string, error)
-	// 处理器有三种情况：
-	// 1. 处理成功，返回处理后的事件
-	// 2. 处理成功，不需要返回处理后端事件，只返回处理结果，将处理结果放到 string 中，比如 eventdrop callback 处理器
-	// 3. 处理失败，返回错误，将错误放到 error 中
+	Init(settings interface{}) (Processor, error)          // 初始化配置
+	Process(ctx *ctx.Context, event *models.AlertCurEvent) // 处理告警事件
 }
 
+// NewProcessorFn 创建处理器的函数类型
 type NewProcessorFn func(settings interface{}) (Processor, error)
 
+// 处理器注册表，存储各种类型处理器的构造函数
 var processorRegister = map[string]NewProcessorFn{}
 
+// // ProcessorTypes 存储所有支持的处理器类型
+// var Processors map[int64]models.Processor
+
+// func init() {
+// 	Processors = make(map[int64]models.Processor)
+// }
+
+// RegisterProcessor 注册处理器类型
 func RegisterProcessor(typ string, p Processor) {
 	if _, found := processorRegister[typ]; found {
 		return
@@ -27,6 +35,7 @@ func RegisterProcessor(typ string, p Processor) {
 	processorRegister[typ] = p.Init
 }
 
+// GetProcessorByType 根据类型获取处理器实例
 func GetProcessorByType(typ string, settings interface{}) (Processor, error) {
 	typ = strings.TrimSpace(typ)
 	fn, found := processorRegister[typ]

alert/pipeline/processor/aisummary/ai_summary.go

@@ -1,198 +0,0 @@
-package aisummary
-
-import (
-	"bytes"
-	"crypto/tls"
-	"encoding/json"
-	"fmt"
-	"io"
-	"net/http"
-	"net/url"
-	"strings"
-	"text/template"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-	"github.com/ccfos/nightingale/v6/pkg/tplx"
-)
-
-const (
-	HTTP_STATUS_SUCCESS_MAX = 299
-)
-
-// AISummaryConfig 配置结构体
-type AISummaryConfig struct {
-	callback.HTTPConfig
-	ModelName      string                 `json:"model_name"`
-	APIKey         string                 `json:"api_key"`
-	PromptTemplate string                 `json:"prompt_template"`
-	CustomParams   map[string]interface{} `json:"custom_params"`
-}
-
-type Message struct {
-	Role    string `json:"role"`
-	Content string `json:"content"`
-}
-
-type ChatCompletionResponse struct {
-	Choices []struct {
-		Message struct {
-			Content string `json:"content"`
-		} `json:"message"`
-	} `json:"choices"`
-}
-
-func init() {
-	models.RegisterProcessor("ai_summary", &AISummaryConfig{})
-}
-
-func (c *AISummaryConfig) Init(settings interface{}) (models.Processor, error) {
-	result, err := common.InitProcessor[*AISummaryConfig](settings)
-	return result, err
-}
-
-func (c *AISummaryConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
-	if c.Client == nil {
-		if err := c.initHTTPClient(); err != nil {
-			return event, "", fmt.Errorf("failed to initialize HTTP client: %v processor: %v", err, c)
-		}
-	}
-
-	// 准备告警事件信息
-	eventInfo, err := c.prepareEventInfo(event)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to prepare event info: %v processor: %v", err, c)
-	}
-
-	// 调用AI模型生成总结
-	summary, err := c.generateAISummary(eventInfo)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to generate AI summary: %v processor: %v", err, c)
-	}
-
-	// 将总结添加到annotations字段
-	if event.AnnotationsJSON == nil {
-		event.AnnotationsJSON = make(map[string]string)
-	}
-	event.AnnotationsJSON["ai_summary"] = summary
-
-	// 更新Annotations字段
-	b, err := json.Marshal(event.AnnotationsJSON)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to marshal annotations: %v processor: %v", err, c)
-	}
-	event.Annotations = string(b)
-
-	return event, "", nil
-}
-
-func (c *AISummaryConfig) initHTTPClient() error {
-	transport := &http.Transport{
-		TLSClientConfig: &tls.Config{InsecureSkipVerify: c.SkipSSLVerify},
-	}
-
-	if c.Proxy != "" {
-		proxyURL, err := url.Parse(c.Proxy)
-		if err != nil {
-			return fmt.Errorf("failed to parse proxy url: %v", err)
-		}
-		transport.Proxy = http.ProxyURL(proxyURL)
-	}
-
-	c.Client = &http.Client{
-		Timeout:   time.Duration(c.Timeout) * time.Millisecond,
-		Transport: transport,
-	}
-	return nil
-}
-
-func (c *AISummaryConfig) prepareEventInfo(event *models.AlertCurEvent) (string, error) {
-	var defs = []string{
-		"{{$event := .}}",
-	}
-
-	text := strings.Join(append(defs, c.PromptTemplate), "")
-	t, err := template.New("prompt").Funcs(template.FuncMap(tplx.TemplateFuncMap)).Parse(text)
-	if err != nil {
-		return "", fmt.Errorf("failed to parse prompt template: %v", err)
-	}
-
-	var body bytes.Buffer
-	err = t.Execute(&body, event)
-	if err != nil {
-		return "", fmt.Errorf("failed to execute prompt template: %v", err)
-	}
-
-	return body.String(), nil
-}
-
-func (c *AISummaryConfig) generateAISummary(eventInfo string) (string, error) {
-	// 构建基础请求参数
-	reqParams := map[string]interface{}{
-		"model": c.ModelName,
-		"messages": []Message{
-			{
-				Role:    "user",
-				Content: eventInfo,
-			},
-		},
-	}
-
-	// 合并自定义参数
-	for k, v := range c.CustomParams {
-		reqParams[k] = v
-	}
-
-	// 序列化请求体
-	jsonData, err := json.Marshal(reqParams)
-	if err != nil {
-		return "", fmt.Errorf("failed to marshal request body: %v", err)
-	}
-
-	// 创建HTTP请求
-	req, err := http.NewRequest("POST", c.URL, bytes.NewBuffer(jsonData))
-	if err != nil {
-		return "", fmt.Errorf("failed to create request: %v", err)
-	}
-
-	// 设置请求头
-	req.Header.Set("Authorization", "Bearer "+c.APIKey)
-	req.Header.Set("Content-Type", "application/json")
-	for k, v := range c.Headers {
-		req.Header.Set(k, v)
-	}
-
-	// 发送请求
-	resp, err := c.Client.Do(req)
-	if err != nil {
-		return "", fmt.Errorf("failed to send request: %v", err)
-	}
-	defer resp.Body.Close()
-
-	// 检查响应状态码
-	if resp.StatusCode > HTTP_STATUS_SUCCESS_MAX {
-		body, _ := io.ReadAll(resp.Body)
-		return "", fmt.Errorf("unexpected status code: %d, body: %s", resp.StatusCode, string(body))
-	}
-
-	// 读取响应
-	body, err := io.ReadAll(resp.Body)
-	if err != nil {
-		return "", fmt.Errorf("failed to read response body: %v", err)
-	}
-
-	// 解析响应
-	var chatResp ChatCompletionResponse
-	if err := json.Unmarshal(body, &chatResp); err != nil {
-		return "", fmt.Errorf("failed to unmarshal response: %v", err)
-	}
-
-	if len(chatResp.Choices) == 0 {
-		return "", fmt.Errorf("no response from AI model")
-	}
-
-	return chatResp.Choices[0].Message.Content, nil
-}

alert/pipeline/processor/aisummary/ai_summary_test.go

@@ -1,69 +0,0 @@
-package aisummary
-
-import (
-	"testing"
-
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-	"github.com/stretchr/testify/assert"
-)
-
-func TestAISummaryConfig_Process(t *testing.T) {
-	// 创建测试配置
-	config := &AISummaryConfig{
-		HTTPConfig: callback.HTTPConfig{
-			URL:           "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions",
-			Timeout:       30000,
-			SkipSSLVerify: true,
-			Headers: map[string]string{
-				"Content-Type": "application/json",
-			},
-		},
-		ModelName:      "gemini-2.0-flash",
-		APIKey:         "*",
-		PromptTemplate: "告警规则：{{$event.RuleName}}\n严重程度：{{$event.Severity}}",
-		CustomParams: map[string]interface{}{
-			"temperature": 0.7,
-			"max_tokens":  2000,
-			"top_p":       0.9,
-		},
-	}
-
-	// 创建测试事件
-	event := &models.AlertCurEvent{
-		RuleName: "Test Rule",
-		Severity: 1,
-		TagsMap: map[string]string{
-			"host": "test-host",
-		},
-		AnnotationsJSON: map[string]string{
-			"description": "Test alert",
-		},
-	}
-
-	// 测试模板处理
-	eventInfo, err := config.prepareEventInfo(event)
-	assert.NoError(t, err)
-	assert.Contains(t, eventInfo, "Test Rule")
-	assert.Contains(t, eventInfo, "1")
-
-	// 测试配置初始化
-	processor, err := config.Init(config)
-	assert.NoError(t, err)
-	assert.NotNil(t, processor)
-
-	// 测试处理函数
-	result, _, err := processor.Process(&ctx.Context{}, event)
-	assert.NoError(t, err)
-	assert.NotNil(t, result)
-	assert.NotEmpty(t, result.AnnotationsJSON["ai_summary"])
-
-	// 展示处理结果
-	t.Log("\n=== 处理结果 ===")
-	t.Logf("告警规则: %s", result.RuleName)
-	t.Logf("严重程度: %d", result.Severity)
-	t.Logf("标签: %v", result.TagsMap)
-	t.Logf("原始注释: %v", result.AnnotationsJSON["description"])
-	t.Logf("AI总结: %s", result.AnnotationsJSON["ai_summary"])
-}

alert/pipeline/processor/callback/callback.go

@@ -1,103 +0,0 @@
-package callback
-
-import (
-	"crypto/tls"
-	"encoding/json"
-	"fmt"
-	"io"
-	"net/http"
-	"net/url"
-	"strings"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-	"github.com/toolkits/pkg/logger"
-)
-
-type HTTPConfig struct {
-	URL           string            `json:"url"`
-	Method        string            `json:"method,omitempty"`
-	Body          string            `json:"body,omitempty"`
-	Headers       map[string]string `json:"header"`
-	AuthUsername  string            `json:"auth_username"`
-	AuthPassword  string            `json:"auth_password"`
-	Timeout       int               `json:"timeout"` // 单位:ms
-	SkipSSLVerify bool              `json:"skip_ssl_verify"`
-	Proxy         string            `json:"proxy"`
-	Client        *http.Client      `json:"-"`
-}
-
-// RelabelConfig
-type CallbackConfig struct {
-	HTTPConfig
-}
-
-func init() {
-	models.RegisterProcessor("callback", &CallbackConfig{})
-}
-
-func (c *CallbackConfig) Init(settings interface{}) (models.Processor, error) {
-	result, err := common.InitProcessor[*CallbackConfig](settings)
-	return result, err
-}
-
-func (c *CallbackConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
-	if c.Client == nil {
-		transport := &http.Transport{
-			TLSClientConfig: &tls.Config{InsecureSkipVerify: c.SkipSSLVerify},
-		}
-
-		if c.Proxy != "" {
-			proxyURL, err := url.Parse(c.Proxy)
-			if err != nil {
-				return event, "", fmt.Errorf("failed to parse proxy url: %v processor: %v", err, c)
-			} else {
-				transport.Proxy = http.ProxyURL(proxyURL)
-			}
-		}
-
-		c.Client = &http.Client{
-			Timeout:   time.Duration(c.Timeout) * time.Millisecond,
-			Transport: transport,
-		}
-	}
-
-	headers := make(map[string]string)
-	headers["Content-Type"] = "application/json"
-	for k, v := range c.Headers {
-		headers[k] = v
-	}
-
-	body, err := json.Marshal(event)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to marshal event: %v processor: %v", err, c)
-	}
-
-	req, err := http.NewRequest("POST", c.URL, strings.NewReader(string(body)))
-	if err != nil {
-		return event, "", fmt.Errorf("failed to create request: %v processor: %v", err, c)
-	}
-
-	for k, v := range headers {
-		req.Header.Set(k, v)
-	}
-
-	if c.AuthUsername != "" && c.AuthPassword != "" {
-		req.SetBasicAuth(c.AuthUsername, c.AuthPassword)
-	}
-
-	resp, err := c.Client.Do(req)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to send request: %v processor: %v", err, c)
-	}
-
-	b, err := io.ReadAll(resp.Body)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to read response body: %v processor: %v", err, c)
-	}
-
-	logger.Debugf("callback processor response body: %s", string(b))
-	return event, "callback success", nil
-}

alert/pipeline/processor/common/common.go

@@ -1,24 +0,0 @@
-package common
-
-import (
-	"encoding/json"
-)
-
-// InitProcessor 是一个通用的初始化处理器的方法
-// 使用泛型简化处理器初始化逻辑
-// T 必须是 models.Processor 接口的实现
-func InitProcessor[T any](settings interface{}) (T, error) {
-	var zero T
-	b, err := json.Marshal(settings)
-	if err != nil {
-		return zero, err
-	}
-
-	var result T
-	err = json.Unmarshal(b, &result)
-	if err != nil {
-		return zero, err
-	}
-
-	return result, nil
-}

alert/pipeline/processor/eventdrop/event_drop.go

@@ -1,60 +0,0 @@
-package eventdrop
-
-import (
-	"bytes"
-	"fmt"
-	"strings"
-	texttemplate "text/template"
-
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-	"github.com/ccfos/nightingale/v6/pkg/tplx"
-	"github.com/toolkits/pkg/logger"
-)
-
-type EventDropConfig struct {
-	Content string `json:"content"`
-}
-
-func init() {
-	models.RegisterProcessor("event_drop", &EventDropConfig{})
-}
-
-func (c *EventDropConfig) Init(settings interface{}) (models.Processor, error) {
-	result, err := common.InitProcessor[*EventDropConfig](settings)
-	return result, err
-}
-
-func (c *EventDropConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
-	// 使用背景是可以根据此处理器，实现对事件进行更加灵活的过滤的逻辑
-	// 在标签过滤和属性过滤都不满足需求时可以使用
-	// 如果模板执行结果为 true，则删除该事件
-
-	var defs = []string{
-		"{{ $event := . }}",
-		"{{ $labels := .TagsMap }}",
-		"{{ $value := .TriggerValue }}",
-	}
-
-	text := strings.Join(append(defs, c.Content), "")
-
-	tpl, err := texttemplate.New("eventdrop").Funcs(tplx.TemplateFuncMap).Parse(text)
-	if err != nil {
-		return event, "", fmt.Errorf("processor failed to parse template: %v processor: %v", err, c)
-	}
-
-	var body bytes.Buffer
-	if err = tpl.Execute(&body, event); err != nil {
-		return event, "", fmt.Errorf("processor failed to execute template: %v processor: %v", err, c)
-	}
-
-	result := strings.TrimSpace(body.String())
-	logger.Infof("processor eventdrop result: %v", result)
-	if result == "true" {
-		logger.Infof("processor eventdrop drop event: %v", event)
-		return event, "drop event success", nil
-	}
-
-	return event, "drop event failed", nil
-}

alert/pipeline/processor/eventupdate/event_update.go

@@ -1,96 +0,0 @@
-package eventupdate
-
-import (
-	"crypto/tls"
-	"encoding/json"
-	"fmt"
-	"io"
-	"net/http"
-	"net/url"
-	"strings"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-	"github.com/toolkits/pkg/logger"
-)
-
-// RelabelConfig
-type EventUpdateConfig struct {
-	callback.HTTPConfig
-}
-
-func init() {
-	models.RegisterProcessor("event_update", &EventUpdateConfig{})
-}
-
-func (c *EventUpdateConfig) Init(settings interface{}) (models.Processor, error) {
-	result, err := common.InitProcessor[*EventUpdateConfig](settings)
-	return result, err
-}
-
-func (c *EventUpdateConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
-	if c.Client == nil {
-		transport := &http.Transport{
-			TLSClientConfig: &tls.Config{InsecureSkipVerify: c.SkipSSLVerify},
-		}
-
-		if c.Proxy != "" {
-			proxyURL, err := url.Parse(c.Proxy)
-			if err != nil {
-				return event, "", fmt.Errorf("failed to parse proxy url: %v processor: %v", err, c)
-			} else {
-				transport.Proxy = http.ProxyURL(proxyURL)
-			}
-		}
-
-		c.Client = &http.Client{
-			Timeout:   time.Duration(c.Timeout) * time.Millisecond,
-			Transport: transport,
-		}
-	}
-
-	headers := make(map[string]string)
-	headers["Content-Type"] = "application/json"
-	for k, v := range c.Headers {
-		headers[k] = v
-	}
-
-	body, err := json.Marshal(event)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to marshal event: %v processor: %v", err, c)
-	}
-
-	req, err := http.NewRequest("POST", c.URL, strings.NewReader(string(body)))
-	if err != nil {
-		return event, "", fmt.Errorf("failed to create request: %v processor: %v", err, c)
-	}
-
-	for k, v := range headers {
-		req.Header.Set(k, v)
-	}
-
-	if c.AuthUsername != "" && c.AuthPassword != "" {
-		req.SetBasicAuth(c.AuthUsername, c.AuthPassword)
-	}
-
-	resp, err := c.Client.Do(req)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to send request: %v processor: %v", err, c)
-	}
-
-	b, err := io.ReadAll(resp.Body)
-	if err != nil {
-		return nil, "", fmt.Errorf("failed to read response body: %v processor: %v", err, c)
-	}
-	logger.Debugf("event update processor response body: %s", string(b))
-
-	err = json.Unmarshal(b, &event)
-	if err != nil {
-		return event, "", fmt.Errorf("failed to unmarshal response body: %v processor: %v", err, c)
-	}
-
-	return event, "", nil
-}

alert/pipeline/processor/relabel/relabel.go

@@ -1,11 +1,12 @@
 package relabel
 
 import (
+	"encoding/json"
 	"fmt"
 	"regexp"
 	"strings"
 
-	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
+	"github.com/ccfos/nightingale/v6/alert/pipeline"
 	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/ctx"
 	"github.com/ccfos/nightingale/v6/pushgw/pconf"
@@ -15,10 +16,6 @@ import (
 	"github.com/prometheus/prometheus/prompb"
 )
 
-const (
-	REPLACE_DOT = "___"
-)
-
 // RelabelConfig
 type RelabelConfig struct {
 	SourceLabels  []string `json:"source_labels"`
@@ -33,16 +30,33 @@ type RelabelConfig struct {
 	Action        string `json:"action"`
 }
 
+func Init() {
+}
+
 func init() {
-	models.RegisterProcessor("relabel", &RelabelConfig{})
+	pipeline.RegisterProcessor("relabel", &RelabelConfig{})
 }
 
-func (r *RelabelConfig) Init(settings interface{}) (models.Processor, error) {
-	result, err := common.InitProcessor[*RelabelConfig](settings)
-	return result, err
+func (r *RelabelConfig) Init(settings interface{}) (pipeline.Processor, error) {
+	b, err := json.Marshal(settings)
+	if err != nil {
+		return nil, err
+	}
+
+	newProcessor := &RelabelConfig{}
+	err = json.Unmarshal(b, &newProcessor)
+	if err != nil {
+		return nil, err
+	}
+
+	return newProcessor, nil
 }
 
-func (r *RelabelConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
+const (
+	REPLACE_DOT = "___"
+)
+
+func (r *RelabelConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) {
 	sourceLabels := make([]model.LabelName, len(r.SourceLabels))
 	for i := range r.SourceLabels {
 		sourceLabels[i] = model.LabelName(strings.ReplaceAll(r.SourceLabels[i], ".", REPLACE_DOT))
@@ -64,7 +78,6 @@ func (r *RelabelConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (
 	}
 
 	EventRelabel(event, relabelConfigs)
-	return event, "", nil
 }
 
 func EventRelabel(event *models.AlertCurEvent, relabelConfigs []*pconf.RelabelConfig) {
@@ -93,6 +106,10 @@ func EventRelabel(event *models.AlertCurEvent, relabelConfigs []*pconf.RelabelCo
 		if relabelConfigs[i].Regex == "" {
 			relabelConfigs[i].Regex = "(.*)"
 		}
+
+		for j := range relabelConfigs[i].SourceLabels {
+			relabelConfigs[i].SourceLabels[j] = model.LabelName(strings.ReplaceAll(string(relabelConfigs[i].SourceLabels[j]), ".", REPLACE_DOT))
+		}
 	}
 
 	gotLabels := writer.Process(labels, relabelConfigs...)

alert/process/process.go

@@ -151,7 +151,7 @@ func (p *Processor) Handle(anomalyPoints []models.AnomalyPoint, from string, inh
 	eventsMap := make(map[string][]*models.AlertCurEvent)
 	for _, anomalyPoint := range anomalyPoints {
 		event := p.BuildEvent(anomalyPoint, from, now, ruleHash)
-		event.NotifyRuleIds = cachedRule.NotifyRuleIds
+		event.NotifyRuleIDs = cachedRule.NotifyRuleIds
 		// 如果 event 被 mute 了,本质也是 fire 的状态,这里无论如何都添加到 alertingKeys 中,防止 fire 的事件自动恢复了
 		hash := event.Hash
 		alertingKeys[hash] = struct{}{}
@@ -280,7 +280,7 @@ func Relabel(rule *models.AlertRule, event *models.AlertCurEvent) {
 
 	// need to keep the original label
 	event.OriginalTags = event.Tags
-	event.OriginalTagsJSON = event.TagsJSON
+	event.OriginalTagsJSON = make([]string, len(event.TagsJSON))
 
 	if len(rule.EventRelabelConfig) == 0 {
 		return
@@ -537,7 +537,7 @@ func (p *Processor) RecoverAlertCurEventFromDb() {
 		if alertRule == nil {
 			continue
 		}
-		event.NotifyRuleIds = alertRule.NotifyRuleIds
+		event.NotifyRuleIDs = alertRule.NotifyRuleIds
 
 		if event.Cate == models.HOST {
 			target, exists := p.TargetCache.Get(event.TargetIdent)

alert/record/prom_rule.go

@@ -56,13 +56,12 @@ func (rrc *RecordRuleContext) Key() string {
 }
 
 func (rrc *RecordRuleContext) Hash() string {
-	return str.MD5(fmt.Sprintf("%d_%s_%s_%d_%s_%s",
+	return str.MD5(fmt.Sprintf("%d_%s_%s_%d_%s",
 		rrc.rule.Id,
 		rrc.rule.CronPattern,
 		rrc.rule.PromQl,
 		rrc.datasourceId,
 		rrc.rule.AppendTags,
-		rrc.rule.Name,
 	))
 }
 

center/cconf/ops.go

@@ -85,221 +85,273 @@ func MergeOperationConf() error {
 const (
 	builtInOps = `
 ops:
-- name: Infrastructure
+- name: dashboards
+  cname: Dashboards
+  ops:
+    - name: "/dashboards"
+      cname: View Dashboards
+    - name: "/dashboards/add"
+      cname: Add Dashboard
+    - name: "/dashboards/put"
+      cname: Modify Dashboard
+    - name: "/dashboards/del"
+      cname: Delete Dashboard
+    - name: "/embedded-dashboards/put"
+      cname: Modify Embedded Dashboard
+    - name: "/embedded-dashboards"
+      cname: View Embedded Dashboard
+    - name: "/public-dashboards"
+      cname: View Public Dashboard
+
+- name: metric
+  cname: Time Series Metrics
+  ops:
+    - name: "/metric/explorer"
+      cname: View Metric Data
+    - name: "/object/explorer"
+      cname: View Object Data
+
+- name: builtin-metrics
+  cname: Metric Views
+  ops:
+    - name: "/metrics-built-in"
+      cname: View Built-in Metrics
+    - name: "/builtin-metrics/add"
+      cname: Add Built-in Metric
+    - name: "/builtin-metrics/put"
+      cname: Modify Built-in Metric
+    - name: "/builtin-metrics/del"
+      cname: Delete Built-in Metric
+
+- name: recording-rules
+  cname: Recording Rule Management
+  ops:
+    - name: "/recording-rules"
+      cname: View Recording Rules
+    - name: "/recording-rules/add"
+      cname: Add Recording Rule
+    - name: "/recording-rules/put"
+      cname: Modify Recording Rule
+    - name: "/recording-rules/del"
+      cname: Delete Recording Rule
+
+- name: log
+  cname: Log Analysis
+  ops:
+    - name: "/log/explorer"
+      cname: View Logs
+    - name: "/log/index-patterns"
+      cname: View Index Patterns
+    - name: "/log/index-patterns/add"
+      cname: Add Index Pattern
+    - name: "/log/index-patterns/put"
+      cname: Modify Index Pattern
+    - name: "/log/index-patterns/del"
+      cname: Delete Index Pattern
+
+
+- name: alert
+  cname: Alert Rules
+  ops:
+    - name: "/alert-rules"
+      cname: View Alert Rules
+    - name: "/alert-rules/add"
+      cname: Add Alert Rule
+    - name: "/alert-rules/put"
+      cname: Modify Alert Rule
+    - name: "/alert-rules/del"
+      cname: Delete Alert Rule
+
+- name: alert-mutes
+  cname: Alert Silence Management
+  ops:
+    - name: "/alert-mutes"
+      cname: View Alert Silences
+    - name: "/alert-mutes/add"
+      cname: Add Alert Silence
+    - name: "/alert-mutes/put"
+      cname: Modify Alert Silence
+    - name: "/alert-mutes/del"
+      cname: Delete Alert Silence
+  
+- name: alert-subscribes
+  cname: Alert Subscription Management
+  ops:
+    - name: "/alert-subscribes"
+      cname: View Alert Subscriptions
+    - name: "/alert-subscribes/add"
+      cname: Add Alert Subscription
+    - name: "/alert-subscribes/put"
+      cname: Modify Alert Subscription
+    - name: "/alert-subscribes/del"
+      cname: Delete Alert Subscription
+
+- name: alert-events  
+  cname: Alert Event Management
+  ops:
+    - name: "/alert-cur-events"
+      cname: View Current Alerts
+    - name: "/alert-cur-events/del"
+      cname: Delete Current Alert
+    - name: "/alert-his-events"
+      cname: View Historical Alerts
+
+- name: notification
+  cname: Alert Notification
+  ops:
+    - name: "/help/notification-settings"
+      cname: View Notification Settings
+    - name: "/help/notification-tpls"
+      cname: View Notification Templates
+
+- name: job
+  cname: Task Management
+  ops:
+    - name: "/job-tpls"
+      cname: View Task Templates
+    - name: "/job-tpls/add"
+      cname: Add Task Template
+    - name: "/job-tpls/put"
+      cname: Modify Task Template
+    - name: "/job-tpls/del"
+      cname: Delete Task Template
+    - name: "/job-tasks"
+      cname: View Task Instances
+    - name: "/job-tasks/add"
+      cname: Add Task Instance
+    - name: "/job-tasks/put"
+      cname: Modify Task Instance
+
+- name: targets
   cname: Infrastructure
   ops:
-    - name: /targets
-      cname: Host - View
-    - name: /targets/put
-      cname: Host - Modify
-    - name: /targets/del
-      cname: Host - Delete
-    - name: /targets/bind
-      cname: Host - Bind Uncategorized
+    - name: "/targets"
+      cname: View Objects
+    - name: "/targets/add"
+      cname: Add Object
+    - name: "/targets/put"
+      cname: Modify Object
+    - name: "/targets/del"
+      cname: Delete Object
+    - name: "/targets/bind"
+      cname: Bind Object
 
-- name: Explorer
-  cname: Explorer
+- name: user
+  cname: User Management
   ops:
-    - name: /metric/explorer
-      cname: Metrics Explorer
-    - name: /object/explorer
-      cname: Quick View
-    - name: /metrics-built-in
-      cname: Built-in Metric - View
-    - name: /builtin-metrics/add
-      cname: Built-in Metric - Add
-    - name: /builtin-metrics/put
-      cname: Built-in Metric - Modify
-    - name: /builtin-metrics/del
-      cname: Built-in Metric - Delete
-    - name: /recording-rules
-      cname: Recording Rule - View
-    - name: /recording-rules/add
-      cname: Recording Rule - Add
-    - name: /recording-rules/put
-      cname: Recording Rule - Modify
-    - name: /recording-rules/del
-      cname: Recording Rule - Delete
-    - name: /log/explorer
-      cname: Logs Explorer
-    - name: /log/index-patterns # 前端有个管理索引模式的页面，所以需要一个权限点来控制，后面应该改成侧拉板
-      cname: Index Pattern - View
-    - name: /log/index-patterns/add
-      cname: Index Pattern - Add
-    - name: /log/index-patterns/put
-      cname: Index Pattern - Modify
-    - name: /log/index-patterns/del
-      cname: Index Pattern - Delete
-    - name: /dashboards
-      cname: Dashboard - View
-    - name: /dashboards/add
-      cname: Dashboard - Add
-    - name: /dashboards/put
-      cname: Dashboard - Modify
-    - name: /dashboards/del
-      cname: Dashboard - Delete
-    - name: /public-dashboards
-      cname: Dashboard - View Public
+    - name: "/users"
+      cname: View User List
+    - name: "/user-groups"
+      cname: View User Groups
+    - name: "/user-groups/add"
+      cname: Add User Group
+    - name: "/user-groups/put"
+      cname: Modify User Group
+    - name: "/user-groups/del"
+      cname: Delete User Group
 
-- name: alerting
-  cname: Alerting
+- name: busi-groups
+  cname: Business Group Management
   ops:
-    - name: /alert-rules
-      cname: Alerting Rule - View
-    - name: /alert-rules/add
-      cname: Alerting Rule - Add
-    - name: /alert-rules/put
-      cname: Alerting Rule - Modify
-    - name: /alert-rules/del
-      cname: Alerting Rule - Delete
-    - name: /alert-mutes
-      cname: Mutting Rule - View
-    - name: /alert-mutes/add
-      cname: Mutting Rule - Add
-    - name: /alert-mutes/put
-      cname: Mutting Rule - Modify
-    - name: /alert-mutes/del
-      cname: Mutting Rule - Delete
-    - name: /alert-subscribes
-      cname: Subscribing Rule - View
-    - name: /alert-subscribes/add
-      cname: Subscribing Rule - Add
-    - name: /alert-subscribes/put
-      cname: Subscribing Rule - Modify
-    - name: /alert-subscribes/del
-      cname: Subscribing Rule - Delete
-    - name: /job-tpls
-      cname: Self-healing-Script - View
-    - name: /job-tpls/add
-      cname: Self-healing-Script - Add
-    - name: /job-tpls/put
-      cname: Self-healing-Script - Modify
-    - name: /job-tpls/del
-      cname: Self-healing-Script - Delete
-    - name: /job-tasks
-      cname: Self-healing-Job - View
-    - name: /job-tasks/add
-      cname: Self-healing-Job - Add
-    - name: /job-tasks/put
-      cname: Self-healing-Job - Modify
-    - name: /alert-cur-events
-      cname: Active Event - View
-    - name: /alert-cur-events/del
-      cname: Active Event - Delete
-    - name: /alert-his-events
-      cname: Historical Event - View
+    - name: "/busi-groups"
+      cname: View Business Groups
+    - name: "/busi-groups/add"
+      cname: Add Business Group
+    - name: "/busi-groups/put"
+      cname: Modify Business Group
+    - name: "/busi-groups/del"
+      cname: Delete Business Group
 
-- name: Notification
-  cname: Notification
+- name: permissions
+  cname: Permission Management
   ops:
-    - name: /notification-rules
-      cname: Notification Rule - View
-    - name: /notification-rules/add
-      cname: Notification Rule - Add
-    - name: /notification-rules/put
-      cname: Notification Rule - Modify
-    - name: /notification-rules/del
-      cname: Notification Rule - Delete
-    - name: /notification-channels
-      cname: Media Type - View
-    - name: /notification-channels/add
-      cname: Media Type - Add
-    - name: /notification-channels/put
-      cname: Media Type - Modify
-    - name: /notification-channels/del
-      cname: Media Type - Delete
-    - name: /notification-templates
-      cname: Message Template - View
-    - name: /notification-templates/add
-      cname: Message Template - Add
-    - name: /notification-templates/put
-      cname: Message Template - Modify
-    - name: /notification-templates/del
-      cname: Message Template - Delete
-    - name: /event-pipelines
-      cname: Event Pipeline - View
-    - name: /event-pipelines/add
-      cname: Event Pipeline - Add
-    - name: /event-pipelines/put
-      cname: Event Pipeline - Modify
-    - name: /event-pipelines/del
-      cname: Event Pipeline - Delete
-    - name: /help/notification-settings # 用于控制老版本的通知设置菜单是否展示
-      cname: Notification Settings - View
-    - name: /help/notification-tpls # 用于控制老版本的通知模板菜单是否展示
-      cname: Notification Templates - View
+    - name: "/permissions"
+      cname: View Permission Settings
+    
+- name: contacts
+  cname: User Contact Management
+  ops:
+    - name: "/contacts"
+      cname: User Contact Management
 
-- name: Integrations
-  cname: Integrations
+- name: built-in-components
+  cname: Template Center
   ops:
-    - name: /datasources # 用于控制能否看到数据源列表页面的菜单。只有 Admin 才能修改、删除数据源
-      cname: Data Source - View
-    - name: /components
-      cname: Component - View
-    - name: /components/add
-      cname: Component - Add
-    - name: /components/put
-      cname: Component - Modify
-    - name: /components/del
-      cname: Component - Delete
-    - name: /embedded-products
-      cname: Embedded Product - View
-    - name: /embedded-product/add
-      cname: Embedded Product - Add
-    - name: /embedded-product/put
-      cname: Embedded Product - Modify
-    - name: /embedded-product/delete
-      cname: Embedded Product - Delete
+    - name: "/built-in-components"
+      cname: View Built-in Components
+    - name: "/built-in-components/add"
+      cname: Add Built-in Component
+    - name: "/built-in-components/put"
+      cname: Modify Built-in Component
+    - name: "/built-in-components/del"
+      cname: Delete Built-in Component
 
-- name: Organization
-  cname: Organization
+- name: datasource
+  cname: Data Source Management
   ops:
-    - name: /users
-      cname: User - View
-    - name: /users/add
-      cname: User - Add
-    - name: /users/put
-      cname: User - Modify
-    - name: /users/del
-      cname: User - Delete
-    - name: /user-groups
-      cname: Team - View
-    - name: /user-groups/add
-      cname: Team - Add
-    - name: /user-groups/put
-      cname: Team - Modify
-    - name: /user-groups/del
-      cname: Team - Delete
-    - name: /busi-groups
-      cname: Business Group - View
-    - name: /busi-groups/add
-      cname: Business Group - Add
-    - name: /busi-groups/put
-      cname: Business Group - Modify
-    - name: /busi-groups/del
-      cname: Business Group - Delete
-    - name: /roles
-      cname: Role - View
-    - name: /roles/add
-      cname: Role - Add
-    - name: /roles/put
-      cname: Role - Modify
-    - name: /roles/del
-      cname: Role - Delete
+    - name: "/help/source"
+      cname: View Data Source Configuration
 
-- name: System Settings
-  cname: System Settings
+- name: system
+  cname: System Information
   ops:
-    - name: /system/site-settings # 仅用于控制能否展示菜单，只有 Admin 才能修改、删除
+    - name: "/help/variable-configs"
+      cname: View Variable Configuration
+    - name: "/help/version"
+      cname: View Version Information
+    - name: "/help/servers"
+      cname: View Server Information
+    - name: "/help/sso"
+      cname: View SSO Configuration
+    - name: "/site-settings"
       cname: View Site Settings
-    - name: /system/variable-settings
-      cname: View Variable Settings
-    - name: /system/sso-settings
-      cname: View SSO Settings
-    - name: /system/alerting-engines
-      cname: View Alerting Engines
-    - name: /system/version
-      cname: View Product Version
 
+- name: message-templates
+  cname: Message Templates
+  ops:
+    - name: "/notification-templates"
+      cname: View Message Templates
+    - name: "/notification-templates/add"
+      cname: Add Message Templates
+    - name: "/notification-templates/put"
+      cname: Modify Message Templates
+    - name: "/notification-templates/del"
+      cname: Delete Message Templates
+
+- name: notify-rules
+  cname: Notify Rules
+  ops:
+    - name: "/notification-rules"
+      cname: View Notify Rules
+    - name: "/notification-rules/add"
+      cname: Add Notify Rules
+    - name: "/notification-rules/put"
+      cname: Modify Notify Rules
+    - name: "/notification-rules/del"
+      cname: Delete Notify Rules
+
+- name: event-pipelines
+  cname: Event Pipelines
+  ops:
+    - name: "/event-pipelines"
+      cname: View Event Pipelines
+    - name: "/event-pipelines/add"
+      cname: Add Event Pipeline
+    - name: "/event-pipelines/put"
+      cname: Modify Event Pipeline
+    - name: "/event-pipelines/del"
+      cname: Delete Event Pipeline
+
+- name: notify-channels
+  cname: Notify Channels
+  ops:
+    - name: "/notification-channels"
+      cname: View Notify Channels
+    - name: "/notification-channels/add"
+      cname: Add Notify Channels
+    - name: "/notification-channels/put"
+      cname: Modify Notify Channels
+    - name: "/notification-channels/del"
+      cname: Delete Notify Channels
 `
 )

center/cconf/plugin.go

@@ -25,22 +25,4 @@ var Plugins = []Plugin{
 		Type:     "tdengine",
 		TypeName: "TDengine",
 	},
-	{
-		Id:       5,
-		Category: "logging",
-		Type:     "ck",
-		TypeName: "ClickHouse",
-	},
-	{
-		Id:       6,
-		Category: "timeseries",
-		Type:     "mysql",
-		TypeName: "MySQL",
-	},
-	{
-		Id:       7,
-		Category: "timeseries",
-		Type:     "pgsql",
-		TypeName: "PostgreSQL",
-	},
 }

center/center.go

@@ -13,6 +13,7 @@ import (
 	alertrt "github.com/ccfos/nightingale/v6/alert/router"
 	"github.com/ccfos/nightingale/v6/center/cconf"
 	"github.com/ccfos/nightingale/v6/center/cconf/rsa"
+	"github.com/ccfos/nightingale/v6/center/cstats"
 	"github.com/ccfos/nightingale/v6/center/integration"
 	"github.com/ccfos/nightingale/v6/center/metas"
 	centerrt "github.com/ccfos/nightingale/v6/center/router"
@@ -59,6 +60,7 @@ func Initialize(configDir string, cryptoKey string) (func(), error) {
 	}
 
 	i18nx.Init(configDir)
+	cstats.Init()
 	flashduty.Init(config.Center.FlashDuty)
 
 	db, err := storage.New(config.DB)
@@ -84,7 +86,7 @@ func Initialize(configDir string, cryptoKey string) (func(), error) {
 	}
 
 	metas := metas.New(redis)
-	idents := idents.New(ctx, redis, config.Pushgw)
+	idents := idents.New(ctx, redis)
 
 	syncStats := memsto.NewSyncStats()
 	alertStats := astats.NewSyncStats()
@@ -92,9 +94,6 @@ func Initialize(configDir string, cryptoKey string) (func(), error) {
 	if config.Center.MigrateBusiGroupLabel || models.CanMigrateBg(ctx) {
 		models.MigrateBg(ctx, config.Pushgw.BusiGroupLabelKey)
 	}
-	if models.CanMigrateEP(ctx) {
-		models.MigrateEP(ctx)
-	}
 
 	configCache := memsto.NewConfigCache(ctx, syncStats, config.HTTP.RSA.RSAPrivateKey, config.HTTP.RSA.RSAPassWord)
 	busiGroupCache := memsto.NewBusiGroupCache(ctx, syncStats)

center/cstats/stats.go

@@ -6,49 +6,40 @@ import (
 	"github.com/prometheus/client_golang/prometheus"
 )
 
-const (
-	namespace = "n9e"
-	subsystem = "center"
-)
+const Service = "n9e-center"
 
 var (
-	uptime = prometheus.NewCounter(
+	labels = []string{"service", "code", "path", "method"}
+
+	uptime = prometheus.NewCounterVec(
 		prometheus.CounterOpts{
-			Namespace: namespace,
-			Subsystem: subsystem,
-			Name:      "uptime",
-			Help:      "HTTP service uptime.",
-		},
+			Name: "uptime",
+			Help: "HTTP service uptime.",
+		}, []string{"service"},
+	)
+
+	RequestCounter = prometheus.NewCounterVec(
+		prometheus.CounterOpts{
+			Name: "http_request_count_total",
+			Help: "Total number of HTTP requests made.",
+		}, labels,
 	)
 
 	RequestDuration = prometheus.NewHistogramVec(
 		prometheus.HistogramOpts{
-			Namespace: namespace,
-			Subsystem: subsystem,
-			Buckets:   prometheus.DefBuckets,
-			Name:      "http_request_duration_seconds",
-			Help:      "HTTP request latencies in seconds.",
-		}, []string{"code", "path", "method"},
-	)
-
-	RedisOperationLatency = prometheus.NewHistogramVec(
-		prometheus.HistogramOpts{
-			Namespace: namespace,
-			Subsystem: subsystem,
-			Name:      "redis_operation_latency_seconds",
-			Help:      "Histogram of latencies for Redis operations",
-			Buckets:   []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5},
-		},
-		[]string{"operation", "status"},
+			Buckets: []float64{.01, .1, 1, 10},
+			Name:    "http_request_duration_seconds",
+			Help:    "HTTP request latencies in seconds.",
+		}, labels,
 	)
 )
 
-func init() {
+func Init() {
 	// Register the summary and the histogram with Prometheus's default registry.
 	prometheus.MustRegister(
 		uptime,
+		RequestCounter,
 		RequestDuration,
-		RedisOperationLatency,
 	)
 
 	go recordUptime()
@@ -57,6 +48,6 @@ func init() {
 // recordUptime increases service uptime per second.
 func recordUptime() {
 	for range time.Tick(time.Second) {
-		uptime.Inc()
+		uptime.WithLabelValues(Service).Inc()
 	}
 }

center/metas/metas.go

@@ -6,7 +6,6 @@ import (
 	"sync"
 	"time"
 
-	"github.com/ccfos/nightingale/v6/center/cstats"
 	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/storage"
 
@@ -116,23 +115,15 @@ func (s *Set) updateTargets(m map[string]models.HostMeta) error {
 		}
 		newMap[models.WrapIdent(ident)] = meta
 	}
-
-	start := time.Now()
 	err := storage.MSet(context.Background(), s.redis, newMap)
 	if err != nil {
-		cstats.RedisOperationLatency.WithLabelValues("mset_target_meta", "fail").Observe(time.Since(start).Seconds())
 		return err
-	} else {
-		cstats.RedisOperationLatency.WithLabelValues("mset_target_meta", "success").Observe(time.Since(start).Seconds())
 	}
 
 	if len(extendMap) > 0 {
 		err = storage.MSet(context.Background(), s.redis, extendMap)
 		if err != nil {
-			cstats.RedisOperationLatency.WithLabelValues("mset_target_extend", "fail").Observe(time.Since(start).Seconds())
 			return err
-		} else {
-			cstats.RedisOperationLatency.WithLabelValues("mset_target_extend", "success").Observe(time.Since(start).Seconds())
 		}
 	}
 

center/router/router.go

@@ -93,9 +93,10 @@ func stat() gin.HandlerFunc {
 
 		code := fmt.Sprintf("%d", c.Writer.Status())
 		method := c.Request.Method
-		labels := []string{code, c.FullPath(), method}
+		labels := []string{cstats.Service, code, c.FullPath(), method}
 
-		cstats.RequestDuration.WithLabelValues(labels...).Observe(time.Since(start).Seconds())
+		cstats.RequestCounter.WithLabelValues(labels...).Inc()
+		cstats.RequestDuration.WithLabelValues(labels...).Observe(float64(time.Since(start).Seconds()))
 	}
 }
 
@@ -264,11 +265,11 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.DELETE("/self/token/:id", rt.auth(), rt.user(), rt.deleteToken)
 
 		pages.GET("/users", rt.auth(), rt.user(), rt.perm("/users"), rt.userGets)
-		pages.POST("/users", rt.auth(), rt.user(), rt.perm("/users/add"), rt.userAddPost)
+		pages.POST("/users", rt.auth(), rt.admin(), rt.userAddPost)
 		pages.GET("/user/:id/profile", rt.auth(), rt.userProfileGet)
-		pages.PUT("/user/:id/profile", rt.auth(), rt.user(), rt.perm("/users/put"), rt.userProfilePut)
-		pages.PUT("/user/:id/password", rt.auth(), rt.user(), rt.perm("/users/put"), rt.userPasswordPut)
-		pages.DELETE("/user/:id", rt.auth(), rt.user(), rt.perm("/users/del"), rt.userDel)
+		pages.PUT("/user/:id/profile", rt.auth(), rt.admin(), rt.userProfilePut)
+		pages.PUT("/user/:id/password", rt.auth(), rt.admin(), rt.userPasswordPut)
+		pages.DELETE("/user/:id", rt.auth(), rt.admin(), rt.userDel)
 
 		pages.GET("/metric-views", rt.auth(), rt.metricViewGets)
 		pages.DELETE("/metric-views", rt.auth(), rt.user(), rt.metricViewDel)
@@ -389,7 +390,6 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.PUT("/busi-group/:id/alert-mute/:amid", rt.auth(), rt.user(), rt.perm("/alert-mutes/put"), rt.alertMutePutByFE)
 		pages.GET("/busi-group/:id/alert-mute/:amid", rt.auth(), rt.user(), rt.perm("/alert-mutes"), rt.alertMuteGet)
 		pages.PUT("/busi-group/:id/alert-mutes/fields", rt.auth(), rt.user(), rt.perm("/alert-mutes/put"), rt.bgrw(), rt.alertMutePutFields)
-		pages.POST("/alert-mute-tryrun", rt.auth(), rt.user(), rt.perm("/alert-mutes/add"), rt.alertMuteTryRun)
 
 		pages.GET("/busi-groups/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes"), rt.alertSubscribeGetsByGids)
 		pages.GET("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes"), rt.bgro(), rt.alertSubscribeGets)
@@ -398,16 +398,21 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.PUT("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/put"), rt.bgrw(), rt.alertSubscribePut)
 		pages.DELETE("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/del"), rt.bgrw(), rt.alertSubscribeDel)
 
-		pages.GET("/alert-cur-event/:eid", rt.alertCurEventGet)
-		pages.GET("/alert-his-event/:eid", rt.alertHisEventGet)
-		pages.GET("/event-notify-records/:eid", rt.notificationRecordList)
+		if rt.Center.AnonymousAccess.AlertDetail {
+			pages.GET("/alert-cur-event/:eid", rt.alertCurEventGet)
+			pages.GET("/alert-his-event/:eid", rt.alertHisEventGet)
+			pages.GET("/event-notify-records/:eid", rt.notificationRecordList)
+		} else {
+			pages.GET("/alert-cur-event/:eid", rt.auth(), rt.user(), rt.alertCurEventGet)
+			pages.GET("/alert-his-event/:eid", rt.auth(), rt.user(), rt.alertHisEventGet)
+			pages.GET("/event-notify-records/:eid", rt.auth(), rt.user(), rt.notificationRecordList)
+		}
 
 		// card logic
 		pages.GET("/alert-cur-events/list", rt.auth(), rt.user(), rt.alertCurEventsList)
 		pages.GET("/alert-cur-events/card", rt.auth(), rt.user(), rt.alertCurEventsCard)
 		pages.POST("/alert-cur-events/card/details", rt.auth(), rt.alertCurEventsCardDetails)
 		pages.GET("/alert-his-events/list", rt.auth(), rt.user(), rt.alertHisEventsList)
-		pages.DELETE("/alert-his-events", rt.auth(), rt.admin(), rt.alertHisEventsDelete)
 		pages.DELETE("/alert-cur-events", rt.auth(), rt.user(), rt.perm("/alert-cur-events/del"), rt.alertCurEventDel)
 		pages.GET("/alert-cur-events/stats", rt.auth(), rt.alertCurEventsStatistics)
 
@@ -439,13 +444,13 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.POST("/datasource/status/update", rt.auth(), rt.admin(), rt.datasourceUpdataStatus)
 		pages.DELETE("/datasource/", rt.auth(), rt.admin(), rt.datasourceDel)
 
-		pages.GET("/roles", rt.auth(), rt.user(), rt.perm("/roles"), rt.roleGets)
-		pages.POST("/roles", rt.auth(), rt.user(), rt.perm("/roles/add"), rt.roleAdd)
-		pages.PUT("/roles", rt.auth(), rt.user(), rt.perm("/roles/put"), rt.rolePut)
-		pages.DELETE("/role/:id", rt.auth(), rt.user(), rt.perm("/roles/del"), rt.roleDel)
+		pages.GET("/roles", rt.auth(), rt.admin(), rt.roleGets)
+		pages.POST("/roles", rt.auth(), rt.admin(), rt.roleAdd)
+		pages.PUT("/roles", rt.auth(), rt.admin(), rt.rolePut)
+		pages.DELETE("/role/:id", rt.auth(), rt.admin(), rt.roleDel)
 
-		pages.GET("/role/:id/ops", rt.auth(), rt.user(), rt.perm("/roles"), rt.operationOfRole)
-		pages.PUT("/role/:id/ops", rt.auth(), rt.user(), rt.perm("/roles/put"), rt.roleBindOperation)
+		pages.GET("/role/:id/ops", rt.auth(), rt.admin(), rt.operationOfRole)
+		pages.PUT("/role/:id/ops", rt.auth(), rt.admin(), rt.roleBindOperation)
 		pages.GET("/operation", rt.operations)
 
 		pages.GET("/notify-tpls", rt.auth(), rt.user(), rt.notifyTplGets)
@@ -467,7 +472,7 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.GET("/notify-channel", rt.auth(), rt.user(), rt.perm("/help/notification-settings"), rt.notifyChannelGets)
 		pages.PUT("/notify-channel", rt.auth(), rt.admin(), rt.notifyChannelPuts)
 
-		pages.GET("/notify-contact", rt.auth(), rt.user(), rt.notifyContactGets)
+		pages.GET("/notify-contact", rt.auth(), rt.user(), rt.perm("/help/notification-settings"), rt.notifyContactGets)
 		pages.PUT("/notify-contact", rt.auth(), rt.admin(), rt.notifyContactPuts)
 
 		pages.GET("/notify-config", rt.auth(), rt.user(), rt.perm("/help/notification-settings"), rt.notifyConfigGet)
@@ -483,13 +488,6 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.GET("/embedded-dashboards", rt.auth(), rt.user(), rt.perm("/embedded-dashboards"), rt.embeddedDashboardsGet)
 		pages.PUT("/embedded-dashboards", rt.auth(), rt.user(), rt.perm("/embedded-dashboards/put"), rt.embeddedDashboardsPut)
 
-		// 获取 embedded-product 列表
-		pages.GET("/embedded-product", rt.auth(), rt.user(), rt.embeddedProductGets)
-		pages.GET("/embedded-product/:id", rt.auth(), rt.user(), rt.embeddedProductGet)
-		pages.POST("/embedded-product", rt.auth(), rt.user(), rt.perm("/embedded-product/add"), rt.embeddedProductAdd)
-		pages.PUT("/embedded-product/:id", rt.auth(), rt.user(), rt.perm("/embedded-product/put"), rt.embeddedProductPut)
-		pages.DELETE("/embedded-product/:id", rt.auth(), rt.user(), rt.perm("/embedded-product/delete"), rt.embeddedProductDelete)
-
 		pages.GET("/user-variable-configs", rt.auth(), rt.user(), rt.perm("/help/variable-configs"), rt.userVariableConfigGets)
 		pages.POST("/user-variable-config", rt.auth(), rt.user(), rt.perm("/help/variable-configs"), rt.userVariableConfigAdd)
 		pages.PUT("/user-variable-config/:id", rt.auth(), rt.user(), rt.perm("/help/variable-configs"), rt.userVariableConfigPut)
@@ -499,23 +497,20 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.PUT("/config", rt.auth(), rt.admin(), rt.configPutByKey)
 		pages.GET("/site-info", rt.siteInfo)
 
-		// source token 相关路由
-		pages.POST("/source-token", rt.auth(), rt.user(), rt.sourceTokenAdd)
-
 		// for admin api
 		pages.GET("/user/busi-groups", rt.auth(), rt.admin(), rt.userBusiGroupsGets)
 
 		pages.GET("/builtin-components", rt.auth(), rt.user(), rt.builtinComponentsGets)
-		pages.POST("/builtin-components", rt.auth(), rt.user(), rt.perm("/components/add"), rt.builtinComponentsAdd)
-		pages.PUT("/builtin-components", rt.auth(), rt.user(), rt.perm("/components/put"), rt.builtinComponentsPut)
-		pages.DELETE("/builtin-components", rt.auth(), rt.user(), rt.perm("/components/del"), rt.builtinComponentsDel)
+		pages.POST("/builtin-components", rt.auth(), rt.user(), rt.perm("/built-in-components/add"), rt.builtinComponentsAdd)
+		pages.PUT("/builtin-components", rt.auth(), rt.user(), rt.perm("/built-in-components/put"), rt.builtinComponentsPut)
+		pages.DELETE("/builtin-components", rt.auth(), rt.user(), rt.perm("/built-in-components/del"), rt.builtinComponentsDel)
 
 		pages.GET("/builtin-payloads", rt.auth(), rt.user(), rt.builtinPayloadsGets)
 		pages.GET("/builtin-payloads/cates", rt.auth(), rt.user(), rt.builtinPayloadcatesGet)
-		pages.POST("/builtin-payloads", rt.auth(), rt.user(), rt.perm("/components/add"), rt.builtinPayloadsAdd)
-		pages.GET("/builtin-payload/:id", rt.auth(), rt.user(), rt.perm("/components"), rt.builtinPayloadGet)
-		pages.PUT("/builtin-payloads", rt.auth(), rt.user(), rt.perm("/components/put"), rt.builtinPayloadsPut)
-		pages.DELETE("/builtin-payloads", rt.auth(), rt.user(), rt.perm("/components/del"), rt.builtinPayloadsDel)
+		pages.POST("/builtin-payloads", rt.auth(), rt.user(), rt.perm("/built-in-components/add"), rt.builtinPayloadsAdd)
+		pages.GET("/builtin-payload/:id", rt.auth(), rt.user(), rt.perm("/built-in-components"), rt.builtinPayloadGet)
+		pages.PUT("/builtin-payloads", rt.auth(), rt.user(), rt.perm("/built-in-components/put"), rt.builtinPayloadsPut)
+		pages.DELETE("/builtin-payloads", rt.auth(), rt.user(), rt.perm("/built-in-components/del"), rt.builtinPayloadsDel)
 		pages.GET("/builtin-payload", rt.auth(), rt.user(), rt.builtinPayloadsGetByUUIDOrID)
 
 		pages.POST("/message-templates", rt.auth(), rt.user(), rt.perm("/notification-templates/add"), rt.messageTemplatesAdd)

center/router/router_alert_cur_event.go

@@ -1,54 +1,50 @@
 package router
 
 import (
-	"fmt"
 	"net/http"
 	"sort"
 	"strings"
 	"time"
 
 	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-	"github.com/ccfos/nightingale/v6/pkg/strx"
 
 	"github.com/gin-gonic/gin"
 	"github.com/toolkits/pkg/ginx"
 )
 
-func getUserGroupIds(ctx *gin.Context, rt *Router, myGroups bool) ([]int64, error) {
-	if !myGroups {
-		return nil, nil
+func parseAggrRules(c *gin.Context) []*models.AggrRule {
+	aggrRules := strings.Split(ginx.QueryStr(c, "rule", ""), "::") // e.g. field:group_name::field:severity::tagkey:ident
+
+	if len(aggrRules) == 0 {
+		ginx.Bomb(http.StatusBadRequest, "rule empty")
 	}
-	me := ctx.MustGet("user").(*models.User)
-	return models.MyGroupIds(rt.Ctx, me.Id)
+
+	rules := make([]*models.AggrRule, len(aggrRules))
+	for i := 0; i < len(aggrRules); i++ {
+		pair := strings.Split(aggrRules[i], ":")
+		if len(pair) != 2 {
+			ginx.Bomb(http.StatusBadRequest, "rule invalid")
+		}
+
+		if !(pair[0] == "field" || pair[0] == "tagkey") {
+			ginx.Bomb(http.StatusBadRequest, "rule invalid")
+		}
+
+		rules[i] = &models.AggrRule{
+			Type:  pair[0],
+			Value: pair[1],
+		}
+	}
+
+	return rules
 }
 
 func (rt *Router) alertCurEventsCard(c *gin.Context) {
 	stime, etime := getTimeRange(c)
-	severity := strx.IdsInt64ForAPI(ginx.QueryStr(c, "severity", ""), ",")
+	severity := ginx.QueryInt(c, "severity", -1)
 	query := ginx.QueryStr(c, "query", "")
-	myGroups := ginx.QueryBool(c, "my_groups", false) // 是否只看自己组，默认false
-
-	var gids []int64
-	var err error
-	if myGroups {
-		gids, err = getUserGroupIds(c, rt, myGroups)
-		ginx.Dangerous(err)
-		if len(gids) == 0 {
-			gids = append(gids, -1)
-		}
-	}
-
-	viewId := ginx.QueryInt64(c, "view_id")
-
-	alertView, err := models.GetAlertAggrViewByViewID(rt.Ctx, viewId)
-	ginx.Dangerous(err)
-
-	if alertView == nil {
-		ginx.Bomb(http.StatusNotFound, "alert aggr view not found")
-	}
-
 	dsIds := queryDatasourceIds(c)
+	rules := parseAggrRules(c)
 
 	prod := ginx.QueryStr(c, "prods", "")
 	if prod == "" {
@@ -65,18 +61,17 @@ func (rt *Router) alertCurEventsCard(c *gin.Context) {
 		cates = strings.Split(cate, ",")
 	}
 
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView, myGroups)
+	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
 	ginx.Dangerous(err)
 
 	// 最多获取50000个，获取太多也没啥意义
 	list, err := models.AlertCurEventsGet(rt.Ctx, prods, bgids, stime, etime, severity, dsIds,
-		cates, 0, query, 50000, 0, []int64{})
+		cates, 0, query, 50000, 0)
 	ginx.Dangerous(err)
 
 	cardmap := make(map[string]*AlertCard)
 	for _, event := range list {
-		title, err := event.GenCardTitle(alertView.Rule)
-		ginx.Dangerous(err)
+		title := event.GenCardTitle(rules)
 		if _, has := cardmap[title]; has {
 			cardmap[title].Total++
 			cardmap[title].EventIds = append(cardmap[title].EventIds, event.Id)
@@ -91,10 +86,6 @@ func (rt *Router) alertCurEventsCard(c *gin.Context) {
 				Severity: event.Severity,
 			}
 		}
-
-		if cardmap[title].Severity < 1 {
-			cardmap[title].Severity = 3
-		}
 	}
 
 	titles := make([]string, 0, len(cardmap))
@@ -151,15 +142,11 @@ func (rt *Router) alertCurEventsGetByRid(c *gin.Context) {
 // 列表方式，拉取活跃告警
 func (rt *Router) alertCurEventsList(c *gin.Context) {
 	stime, etime := getTimeRange(c)
-	severity := strx.IdsInt64ForAPI(ginx.QueryStr(c, "severity", ""), ",")
+	severity := ginx.QueryInt(c, "severity", -1)
 	query := ginx.QueryStr(c, "query", "")
 	limit := ginx.QueryInt(c, "limit", 20)
-	myGroups := ginx.QueryBool(c, "my_groups", false) // 是否只看自己组，默认false
-
 	dsIds := queryDatasourceIds(c)
 
-	eventIds := strx.IdsInt64ForAPI(ginx.QueryStr(c, "event_ids", ""), ",")
-
 	prod := ginx.QueryStr(c, "prods", "")
 	if prod == "" {
 		prod = ginx.QueryStr(c, "rule_prods", "")
@@ -178,19 +165,18 @@ func (rt *Router) alertCurEventsList(c *gin.Context) {
 
 	ruleId := ginx.QueryInt64(c, "rid", 0)
 
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView, myGroups)
+	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
 	ginx.Dangerous(err)
 
 	total, err := models.AlertCurEventTotal(rt.Ctx, prods, bgids, stime, etime, severity, dsIds,
-		cates, ruleId, query, eventIds)
+		cates, ruleId, query)
 	ginx.Dangerous(err)
 
 	list, err := models.AlertCurEventsGet(rt.Ctx, prods, bgids, stime, etime, severity, dsIds,
-		cates, ruleId, query, limit, ginx.Offset(c, limit), eventIds)
+		cates, ruleId, query, limit, ginx.Offset(c, limit))
 	ginx.Dangerous(err)
 
 	cache := make(map[int64]*models.UserGroup)
-
 	for i := 0; i < len(list); i++ {
 		list[i].FillNotifyGroups(rt.Ctx, cache)
 	}
@@ -232,68 +218,24 @@ func (rt *Router) checkCurEventBusiGroupRWPermission(c *gin.Context, ids []int64
 
 func (rt *Router) alertCurEventGet(c *gin.Context) {
 	eid := ginx.UrlParamInt64(c, "eid")
-	event, err := GetCurEventDetail(rt.Ctx, eid)
+	event, err := models.AlertCurEventGetById(rt.Ctx, eid)
+	ginx.Dangerous(err)
 
-	hasPermission := HasPermission(rt.Ctx, c, "event", fmt.Sprintf("%d", eid), rt.Center.AnonymousAccess.AlertDetail)
-	if !hasPermission {
-		rt.auth()(c)
-		rt.user()(c)
+	if event == nil {
+		ginx.Bomb(404, "No such active event")
+	}
+
+	if !rt.Center.AnonymousAccess.AlertDetail && rt.Center.EventHistoryGroupView {
 		rt.bgroCheck(c, event.GroupId)
 	}
 
-	ginx.NewRender(c).Data(event, err)
-}
-
-func GetCurEventDetail(ctx *ctx.Context, eid int64) (*models.AlertCurEvent, error) {
-	event, err := models.AlertCurEventGetById(ctx, eid)
-	if err != nil {
-		return nil, err
-	}
-
-	if event == nil {
-		return nil, fmt.Errorf("no such active event")
-	}
-
-	ruleConfig, needReset := models.FillRuleConfigTplName(ctx, event.RuleConfig)
+	ruleConfig, needReset := models.FillRuleConfigTplName(rt.Ctx, event.RuleConfig)
 	if needReset {
 		event.RuleConfigJson = ruleConfig
 	}
 
 	event.LastEvalTime = event.TriggerTime
-	event.NotifyVersion, err = GetEventNotifyVersion(ctx, event.RuleId, event.NotifyRuleIds)
-	ginx.Dangerous(err)
-
-	event.NotifyRules, err = GetEventNorifyRuleNames(ctx, event.NotifyRuleIds)
-	return event, err
-}
-
-func GetEventNorifyRuleNames(ctx *ctx.Context, notifyRuleIds []int64) ([]*models.EventNotifyRule, error) {
-	notifyRuleNames := make([]*models.EventNotifyRule, 0)
-	notifyRules, err := models.NotifyRulesGet(ctx, "id in ?", notifyRuleIds)
-	if err != nil {
-		return nil, err
-	}
-
-	for _, notifyRule := range notifyRules {
-		notifyRuleNames = append(notifyRuleNames, &models.EventNotifyRule{
-			Id:   notifyRule.ID,
-			Name: notifyRule.Name,
-		})
-	}
-	return notifyRuleNames, nil
-}
-
-func GetEventNotifyVersion(ctx *ctx.Context, ruleId int64, notifyRuleIds []int64) (int, error) {
-	if len(notifyRuleIds) != 0 {
-		// 如果存在 notify_rule_ids，则认为使用新的告警通知方式
-		return 1, nil
-	}
-
-	rule, err := models.AlertRuleGetById(ctx, ruleId)
-	if err != nil {
-		return 0, err
-	}
-	return rule.NotifyVersion, nil
+	ginx.NewRender(c).Data(event, nil)
 }
 
 func (rt *Router) alertCurEventsStatistics(c *gin.Context) {

center/router/router_alert_his_event.go

@@ -2,7 +2,6 @@ package router
 
 import (
 	"fmt"
-	"net/http"
 	"strings"
 	"time"
 
@@ -11,7 +10,6 @@ import (
 
 	"github.com/gin-gonic/gin"
 	"github.com/toolkits/pkg/ginx"
-	"github.com/toolkits/pkg/logger"
 	"golang.org/x/exp/slices"
 )
 
@@ -58,7 +56,7 @@ func (rt *Router) alertHisEventsList(c *gin.Context) {
 
 	ruleId := ginx.QueryInt64(c, "rid", 0)
 
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView, false)
+	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
 	ginx.Dangerous(err)
 
 	total, err := models.AlertHisEventTotal(rt.Ctx, prods, bgids, stime, etime, severity,
@@ -80,56 +78,16 @@ func (rt *Router) alertHisEventsList(c *gin.Context) {
 	}, nil)
 }
 
-type alertHisEventsDeleteForm struct {
-	Severities []int `json:"severities"`
-	Timestamp  int64 `json:"timestamp" binding:"required"`
-}
-
-func (rt *Router) alertHisEventsDelete(c *gin.Context) {
-	var f alertHisEventsDeleteForm
-	ginx.BindJSON(c, &f)
-	// 校验
-	if f.Timestamp == 0 {
-		ginx.Bomb(http.StatusBadRequest, "timestamp parameter is required")
-		return
-	}
-
-	user := c.MustGet("user").(*models.User)
-
-	// 启动后台清理任务
-	go func() {
-		limit := 100
-		for {
-			n, err := models.AlertHisEventBatchDelete(rt.Ctx, f.Timestamp, f.Severities, limit)
-			if err != nil {
-				logger.Errorf("Failed to delete alert history events: operator=%s, timestamp=%d, severities=%v, error=%v",
-					user.Username, f.Timestamp, f.Severities, err)
-				break
-			}
-			logger.Debugf("Successfully deleted alert history events: operator=%s, timestamp=%d, severities=%v, deleted=%d",
-				user.Username, f.Timestamp, f.Severities, n)
-			if n < int64(limit) {
-				break // 已经删完
-			}
-
-			time.Sleep(100 * time.Millisecond) // 防止锁表
-		}
-	}()
-	ginx.NewRender(c).Message("Alert history events deletion started")
-}
-
 func (rt *Router) alertHisEventGet(c *gin.Context) {
 	eid := ginx.UrlParamInt64(c, "eid")
 	event, err := models.AlertHisEventGetById(rt.Ctx, eid)
 	ginx.Dangerous(err)
+
 	if event == nil {
 		ginx.Bomb(404, "No such alert event")
 	}
 
-	hasPermission := HasPermission(rt.Ctx, c, "event", fmt.Sprintf("%d", eid), rt.Center.AnonymousAccess.AlertDetail)
-	if !hasPermission {
-		rt.auth()(c)
-		rt.user()(c)
+	if !rt.Center.AnonymousAccess.AlertDetail && rt.Center.EventHistoryGroupView {
 		rt.bgroCheck(c, event.GroupId)
 	}
 
@@ -138,54 +96,46 @@ func (rt *Router) alertHisEventGet(c *gin.Context) {
 		event.RuleConfigJson = ruleConfig
 	}
 
-	event.NotifyVersion, err = GetEventNotifyVersion(rt.Ctx, event.RuleId, event.NotifyRuleIds)
-	ginx.Dangerous(err)
-
-	event.NotifyRules, err = GetEventNorifyRuleNames(rt.Ctx, event.NotifyRuleIds)
 	ginx.NewRender(c).Data(event, err)
 }
 
-func GetBusinessGroupIds(c *gin.Context, ctx *ctx.Context, onlySelfGroupView bool, myGroups bool) ([]int64, error) {
+func GetBusinessGroupIds(c *gin.Context, ctx *ctx.Context, eventHistoryGroupView bool) ([]int64, error) {
 	bgid := ginx.QueryInt64(c, "bgid", 0)
 	var bgids []int64
 
-	if strings.HasPrefix(c.Request.URL.Path, "/v1") {
-		// 如果请求路径以 /v1 开头，不查询用户信息
+	if !eventHistoryGroupView || strings.HasPrefix(c.Request.URL.Path, "/v1") {
 		if bgid > 0 {
 			return []int64{bgid}, nil
 		}
-
 		return bgids, nil
 	}
 
 	user := c.MustGet("user").(*models.User)
-	if myGroups || (onlySelfGroupView && !user.IsAdmin()) {
-		// 1. 页面上勾选了我的业务组，需要查询用户所属的业务组
-		// 2. 如果 onlySelfGroupView 为 true，表示只允许查询用户所属的业务组
-		bussGroupIds, err := models.MyBusiGroupIds(ctx, user.Id)
-		if err != nil {
-			return nil, err
-		}
-
-		if len(bussGroupIds) == 0 {
-			// 如果没查到用户属于任何业务组，需要返回一个0，否则会导致查询到全部告警历史
-			return []int64{0}, nil
-		}
-
+	if user.IsAdmin() {
 		if bgid > 0 {
-			if !slices.Contains(bussGroupIds, bgid) && !user.IsAdmin() {
-				return nil, fmt.Errorf("business group ID not allowed")
-			}
-
 			return []int64{bgid}, nil
 		}
+		return bgids, nil
+	}
 
-		return bussGroupIds, nil
+	bussGroupIds, err := models.MyBusiGroupIds(ctx, user.Id)
+	if err != nil {
+		return nil, err
+	}
+
+	if len(bussGroupIds) == 0 {
+		// 如果没查到用户属于任何业务组，需要返回一个0，否则会导致查询到全部告警历史
+		return []int64{0}, nil
+	}
+
+	if bgid > 0 && !slices.Contains(bussGroupIds, bgid) {
+		return nil, fmt.Errorf("business group ID not allowed")
 	}
 
 	if bgid > 0 {
+		// Pass filter parameters, priority to use
 		return []int64{bgid}, nil
 	}
 
-	return bgids, nil
+	return bussGroupIds, nil
 }

center/router/router_builtin_metric_filter.go

@@ -57,7 +57,7 @@ func (rt *Router) metricFilterDel(c *gin.Context) {
 			ginx.Dangerous(err)
 
 			if !HasPerm(gids, old.GroupsPerm, true) {
-				ginx.NewRender(c).Message("forbidden")
+				ginx.NewRender(c).Message("no permission")
 				return
 			}
 		}
@@ -79,7 +79,7 @@ func (rt *Router) metricFilterPut(c *gin.Context) {
 		ginx.Dangerous(err)
 
 		if !HasPerm(gids, old.GroupsPerm, true) {
-			ginx.NewRender(c).Message("forbidden")
+			ginx.NewRender(c).Message("no permission")
 			return
 		}
 	}

center/router/router_builtin_metrics.go

@@ -86,11 +86,15 @@ func (rt *Router) builtinMetricsDel(c *gin.Context) {
 func (rt *Router) builtinMetricsDefaultTypes(c *gin.Context) {
 	lst := []string{
 		"Linux",
-		"Procstat",
 		"cAdvisor",
 		"Ping",
 		"MySQL",
-		"ClickHouse",
+		"Redis",
+		"Kafka",
+		"Elasticsearch",
+		"PostgreSQL",
+		"MongoDB",
+		"Memcached",
 	}
 	ginx.NewRender(c).Data(lst, nil)
 }
@@ -98,10 +102,29 @@ func (rt *Router) builtinMetricsDefaultTypes(c *gin.Context) {
 func (rt *Router) builtinMetricsTypes(c *gin.Context) {
 	collector := ginx.QueryStr(c, "collector", "")
 	query := ginx.QueryStr(c, "query", "")
+	disabled := ginx.QueryInt(c, "disabled", -1)
 	lang := c.GetHeader("X-Language")
 
 	metricTypeList, err := models.BuiltinMetricTypes(rt.Ctx, lang, collector, query)
-	ginx.NewRender(c).Data(metricTypeList, err)
+	ginx.Dangerous(err)
+
+	componentList, err := models.BuiltinComponentGets(rt.Ctx, "", disabled)
+	ginx.Dangerous(err)
+
+	// 创建一个 map 来存储 componentList 中的类型
+	componentTypes := make(map[string]struct{})
+	for _, comp := range componentList {
+		componentTypes[comp.Ident] = struct{}{}
+	}
+
+	filteredMetricTypeList := make([]string, 0)
+	for _, metricType := range metricTypeList {
+		if _, exists := componentTypes[metricType]; exists {
+			filteredMetricTypeList = append(filteredMetricTypeList, metricType)
+		}
+	}
+
+	ginx.NewRender(c).Data(filteredMetricTypeList, nil)
 }
 
 func (rt *Router) builtinMetricsCollectors(c *gin.Context) {

center/router/router_datasource.go

@@ -148,12 +148,11 @@ func DatasourceCheck(ds models.Datasource) error {
 		},
 	}
 
-	ds.HTTPJson.Url = strings.TrimRight(ds.HTTPJson.Url, "/")
 	var fullURL string
 	req, err := ds.HTTPJson.NewReq(&fullURL)
 	if err != nil {
 		logger.Errorf("Error creating request: %v", err)
-		return fmt.Errorf("request urls:%v failed: %v", ds.HTTPJson.GetUrls(), err)
+		return fmt.Errorf("request urls:%v failed", ds.HTTPJson.GetUrls())
 	}
 
 	if ds.PluginType == models.PROMETHEUS {
@@ -169,14 +168,14 @@ func DatasourceCheck(ds models.Datasource) error {
 		req, err = http.NewRequest("GET", fullURL, nil)
 		if err != nil {
 			logger.Errorf("Error creating request: %v", err)
-			return fmt.Errorf("request url:%s failed: %v", fullURL, err)
+			return fmt.Errorf("request url:%s failed", fullURL)
 		}
 	} else if ds.PluginType == models.TDENGINE {
 		fullURL = fmt.Sprintf("%s/rest/sql", ds.HTTPJson.Url)
 		req, err = http.NewRequest("POST", fullURL, strings.NewReader("show databases"))
 		if err != nil {
 			logger.Errorf("Error creating request: %v", err)
-			return fmt.Errorf("request url:%s failed: %v", fullURL, err)
+			return fmt.Errorf("request url:%s failed", fullURL)
 		}
 	}
 
@@ -188,7 +187,7 @@ func DatasourceCheck(ds models.Datasource) error {
 		req, err = http.NewRequest("GET", fullURL, nil)
 		if err != nil {
 			logger.Errorf("Error creating request: %v", err)
-			return fmt.Errorf("request url:%s failed: %v", fullURL, err)
+			return fmt.Errorf("request url:%s failed", fullURL)
 		}
 	}
 
@@ -203,7 +202,7 @@ func DatasourceCheck(ds models.Datasource) error {
 	resp, err := client.Do(req)
 	if err != nil {
 		logger.Errorf("Error making request: %v\n", err)
-		return fmt.Errorf("request url:%s failed: %v", fullURL, err)
+		return fmt.Errorf("request url:%s failed", fullURL)
 	}
 	defer resp.Body.Close()
 

center/router/router_embedded.go

@@ -1,141 +0,0 @@
-package router
-
-import (
-	"time"
-
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-
-	"github.com/gin-gonic/gin"
-	"github.com/toolkits/pkg/ginx"
-)
-
-func (rt *Router) embeddedProductGets(c *gin.Context) {
-	products, err := models.EmbeddedProductGets(rt.Ctx)
-	ginx.Dangerous(err)
-	// 获取当前用户可访问的Group ID 列表
-	me := c.MustGet("user").(*models.User)
-
-	if me.IsAdmin() {
-		ginx.NewRender(c).Data(products, err)
-		return
-	}
-
-	gids, err := models.MyGroupIds(rt.Ctx, me.Id)
-	bgSet := make(map[int64]struct{}, len(gids))
-	for _, id := range gids {
-		bgSet[id] = struct{}{}
-	}
-
-	// 过滤出公开或有权限访问的私有 product link
-	var result []*models.EmbeddedProduct
-	for _, product := range products {
-		if !product.IsPrivate {
-			result = append(result, product)
-			continue
-		}
-
-		for _, tid := range product.TeamIDs {
-			if _, ok := bgSet[tid]; ok {
-				result = append(result, product)
-				break
-			}
-		}
-	}
-
-	ginx.NewRender(c).Data(result, err)
-}
-
-func (rt *Router) embeddedProductGet(c *gin.Context) {
-	id := ginx.UrlParamInt64(c, "id")
-	if id <= 0 {
-		ginx.Bomb(400, "invalid id")
-	}
-
-	data, err := models.GetEmbeddedProductByID(rt.Ctx, id)
-	ginx.Dangerous(err)
-
-	me := c.MustGet("user").(*models.User)
-	hashPermission, err := hasEmbeddedProductAccess(rt.Ctx, me, data)
-	ginx.Dangerous(err)
-
-	if !hashPermission {
-		ginx.Bomb(403, "forbidden")
-	}
-
-	ginx.NewRender(c).Data(data, nil)
-}
-
-func (rt *Router) embeddedProductAdd(c *gin.Context) {
-	var eps []models.EmbeddedProduct
-	ginx.BindJSON(c, &eps)
-
-	me := c.MustGet("user").(*models.User)
-
-	for i := range eps {
-		eps[i].CreateBy = me.Nickname
-		eps[i].UpdateBy = me.Nickname
-	}
-
-	err := models.AddEmbeddedProduct(rt.Ctx, eps)
-	ginx.NewRender(c).Message(err)
-}
-
-func (rt *Router) embeddedProductPut(c *gin.Context) {
-	var ep models.EmbeddedProduct
-	id := ginx.UrlParamInt64(c, "id")
-	ginx.BindJSON(c, &ep)
-
-	if id <= 0 {
-		ginx.Bomb(400, "invalid id")
-	}
-
-	oldProduct, err := models.GetEmbeddedProductByID(rt.Ctx, id)
-	ginx.Dangerous(err)
-	me := c.MustGet("user").(*models.User)
-
-	now := time.Now().Unix()
-	oldProduct.Name = ep.Name
-	oldProduct.URL = ep.URL
-	oldProduct.IsPrivate = ep.IsPrivate
-	oldProduct.TeamIDs = ep.TeamIDs
-	oldProduct.UpdateBy = me.Username
-	oldProduct.UpdateAt = now
-
-	err = models.UpdateEmbeddedProduct(rt.Ctx, oldProduct)
-	ginx.NewRender(c).Message(err)
-}
-
-func (rt *Router) embeddedProductDelete(c *gin.Context) {
-	id := ginx.UrlParamInt64(c, "id")
-	if id <= 0 {
-		ginx.Bomb(400, "invalid id")
-	}
-
-	err := models.DeleteEmbeddedProduct(rt.Ctx, id)
-	ginx.NewRender(c).Message(err)
-}
-
-func hasEmbeddedProductAccess(ctx *ctx.Context, user *models.User, ep *models.EmbeddedProduct) (bool, error) {
-	if user.IsAdmin() || !ep.IsPrivate {
-		return true, nil
-	}
-
-	gids, err := models.MyGroupIds(ctx, user.Id)
-	if err != nil {
-		return false, err
-	}
-
-	groupSet := make(map[int64]struct{}, len(gids))
-	for _, gid := range gids {
-		groupSet[gid] = struct{}{}
-	}
-
-	for _, tid := range ep.TeamIDs {
-		if _, ok := groupSet[tid]; ok {
-			return true, nil
-		}
-	}
-
-	return false, nil
-}

center/router/router_event_pipeline.go

@@ -4,8 +4,8 @@ import (
 	"net/http"
 	"time"
 
+	"github.com/ccfos/nightingale/v6/alert/pipeline"
 	"github.com/ccfos/nightingale/v6/models"
-
 	"github.com/gin-gonic/gin"
 	"github.com/toolkits/pkg/ginx"
 )
@@ -139,23 +139,12 @@ func (rt *Router) tryRunEventPipeline(c *gin.Context) {
 	}
 	event := hisEvent.ToCur()
 
-	for _, p := range f.PipelineConfig.ProcessorConfigs {
-		processor, err := models.GetProcessorByType(p.Typ, p.Config)
+	for _, p := range f.PipelineConfig.Processors {
+		processor, err := pipeline.GetProcessorByType(p.Typ, p.Config)
 		if err != nil {
-			ginx.Bomb(http.StatusBadRequest, "get processor: %+v err: %+v", p, err)
-		}
-		event, _, err = processor.Process(rt.Ctx, event)
-		if err != nil {
-			ginx.Bomb(http.StatusBadRequest, "processor: %+v err: %+v", p, err)
-		}
-
-		if event == nil {
-			ginx.NewRender(c).Data(map[string]interface{}{
-				"event":  event,
-				"result": "event is dropped",
-			}, nil)
-			return
+			ginx.Bomb(http.StatusBadRequest, "processor %+v type not found, error: %s", p, err.Error())
 		}
+		processor.Process(rt.Ctx, event)
 	}
 
 	ginx.NewRender(c).Data(event, nil)
@@ -164,8 +153,8 @@ func (rt *Router) tryRunEventPipeline(c *gin.Context) {
 // 测试事件处理器
 func (rt *Router) tryRunEventProcessor(c *gin.Context) {
 	var f struct {
-		EventId         int64                  `json:"event_id"`
-		ProcessorConfig models.ProcessorConfig `json:"processor_config"`
+		EventId         int64            `json:"event_id"`
+		ProcessorConfig models.Processor `json:"processor_config"`
 	}
 	ginx.BindJSON(c, &f)
 
@@ -175,19 +164,13 @@ func (rt *Router) tryRunEventProcessor(c *gin.Context) {
 	}
 	event := hisEvent.ToCur()
 
-	processor, err := models.GetProcessorByType(f.ProcessorConfig.Typ, f.ProcessorConfig.Config)
+	processor, err := pipeline.GetProcessorByType(f.ProcessorConfig.Typ, f.ProcessorConfig.Config)
 	if err != nil {
-		ginx.Bomb(http.StatusBadRequest, "get processor err: %+v", err)
-	}
-	event, res, err := processor.Process(rt.Ctx, event)
-	if err != nil {
-		ginx.Bomb(http.StatusBadRequest, "processor err: %+v", err)
+		ginx.Bomb(http.StatusBadRequest, "processor type: %s, error: %s", f.ProcessorConfig.Typ, err.Error())
 	}
+	processor.Process(rt.Ctx, event)
 
-	ginx.NewRender(c).Data(map[string]interface{}{
-		"event":  event,
-		"result": res,
-	}, nil)
+	ginx.NewRender(c).Data(event, nil)
 }
 
 func (rt *Router) tryRunEventProcessorByNotifyRule(c *gin.Context) {
@@ -216,23 +199,12 @@ func (rt *Router) tryRunEventProcessorByNotifyRule(c *gin.Context) {
 	}
 
 	for _, pl := range pipelines {
-		for _, p := range pl.ProcessorConfigs {
-			processor, err := models.GetProcessorByType(p.Typ, p.Config)
+		for _, p := range pl.Processors {
+			processor, err := pipeline.GetProcessorByType(p.Typ, p.Config)
 			if err != nil {
-				ginx.Bomb(http.StatusBadRequest, "get processor: %+v err: %+v", p, err)
-			}
-
-			event, _, err := processor.Process(rt.Ctx, event)
-			if err != nil {
-				ginx.Bomb(http.StatusBadRequest, "processor: %+v err: %+v", p, err)
-			}
-			if event == nil {
-				ginx.NewRender(c).Data(map[string]interface{}{
-					"event":  event,
-					"result": "event is dropped",
-				}, nil)
-				return
+				ginx.Bomb(http.StatusBadRequest, "processor %+v type not found", p)
 			}
+			processor.Process(rt.Ctx, event)
 		}
 	}
 

center/router/router_funcs.go

@@ -173,38 +173,3 @@ func Username(c *gin.Context) string {
 	}
 	return username
 }
-
-func HasPermission(ctx *ctx.Context, c *gin.Context, sourceType, sourceId string, isAnonymousAccess bool) bool {
-	if sourceType == "event" && isAnonymousAccess {
-		return true
-	}
-
-	// 尝试从请求中获取 __token 参数
-	token := ginx.QueryStr(c, "__token", "")
-
-	// 如果有 __token 参数，验证其合法性
-	if token != "" {
-		return ValidateSourceToken(ctx, sourceType, sourceId, token)
-	}
-
-	return false
-}
-
-func ValidateSourceToken(ctx *ctx.Context, sourceType, sourceId, token string) bool {
-	if token == "" {
-		return false
-	}
-
-	// 根据源类型、源ID和令牌获取源令牌记录
-	sourceToken, err := models.GetSourceTokenBySource(ctx, sourceType, sourceId, token)
-	if err != nil {
-		return false
-	}
-
-	// 检查令牌是否过期
-	if sourceToken.IsExpired() {
-		return false
-	}
-
-	return true
-}

center/router/router_message_template.go

@@ -12,9 +12,7 @@ import (
 	"github.com/ccfos/nightingale/v6/pkg/slice"
 	"github.com/ccfos/nightingale/v6/pkg/strx"
 	"github.com/ccfos/nightingale/v6/pkg/tplx"
-
 	"github.com/gin-gonic/gin"
-	"github.com/google/uuid"
 	"github.com/toolkits/pkg/ginx"
 )
 
@@ -32,12 +30,9 @@ func (rt *Router) messageTemplatesAdd(c *gin.Context) {
 	ginx.Dangerous(err)
 	now := time.Now().Unix()
 	for _, tpl := range lst {
-		// 生成一个唯一的标识符，以后也不允许修改，前端不需要传这个参数
-		tpl.Ident = uuid.New().String()
-
 		ginx.Dangerous(tpl.Verify())
 		if !isAdmin && !slice.HaveIntersection(gids, tpl.UserGroupIds) {
-			ginx.Bomb(http.StatusForbidden, "forbidden")
+			ginx.Bomb(http.StatusForbidden, "no permission")
 		}
 		idents = append(idents, tpl.Ident)
 
@@ -80,8 +75,8 @@ func (rt *Router) messageTemplatesDel(c *gin.Context) {
 		gids, err := models.MyGroupIds(rt.Ctx, me.Id)
 		ginx.Dangerous(err)
 		for _, t := range lst {
-			if !slice.HaveIntersection(gids, t.UserGroupIds) {
-				ginx.Bomb(http.StatusForbidden, "forbidden")
+			if !slice.HaveIntersection[int64](gids, t.UserGroupIds) {
+				ginx.Bomb(http.StatusForbidden, "no permission")
 			}
 		}
 	}
@@ -110,8 +105,8 @@ func (rt *Router) messageTemplatePut(c *gin.Context) {
 	if !me.IsAdmin() {
 		gids, err := models.MyGroupIds(rt.Ctx, me.Id)
 		ginx.Dangerous(err)
-		if !slice.HaveIntersection(gids, mt.UserGroupIds) {
-			ginx.Bomb(http.StatusForbidden, "forbidden")
+		if !slice.HaveIntersection[int64](gids, mt.UserGroupIds) {
+			ginx.Bomb(http.StatusForbidden, "no permission")
 		}
 	}
 
@@ -130,8 +125,8 @@ func (rt *Router) messageTemplateGet(c *gin.Context) {
 	if mt == nil {
 		ginx.Bomb(http.StatusNotFound, "message template not found")
 	}
-	if mt.Private == 1 && !slice.HaveIntersection(gids, mt.UserGroupIds) {
-		ginx.Bomb(http.StatusForbidden, "forbidden")
+	if mt.Private == 1 && !slice.HaveIntersection[int64](gids, mt.UserGroupIds) {
+		ginx.Bomb(http.StatusForbidden, "no permission")
 	}
 
 	ginx.NewRender(c).Data(mt, nil)

center/router/router_mute.go

@@ -1,13 +1,11 @@
 package router
 
 import (
-	"math"
 	"net/http"
 	"strings"
 	"time"
 
 	"github.com/ccfos/nightingale/v6/alert/common"
-	"github.com/ccfos/nightingale/v6/alert/mute"
 	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/strx"
 
@@ -65,45 +63,10 @@ func (rt *Router) alertMuteAdd(c *gin.Context) {
 
 	username := c.MustGet("username").(string)
 	f.CreateBy = username
-	f.UpdateBy = username
 	f.GroupId = ginx.UrlParamInt64(c, "id")
 	ginx.NewRender(c).Message(f.Add(rt.Ctx))
 }
 
-type MuteTestForm struct {
-	EventId   int64            `json:"event_id" binding:"required"`
-	AlertMute models.AlertMute `json:"mute_config" binding:"required"`
-}
-
-func (rt *Router) alertMuteTryRun(c *gin.Context) {
-
-	var f MuteTestForm
-	ginx.BindJSON(c, &f)
-
-	hisEvent, err := models.AlertHisEventGetById(rt.Ctx, f.EventId)
-	ginx.Dangerous(err)
-
-	if hisEvent == nil {
-		ginx.Bomb(http.StatusNotFound, "event not found")
-	}
-
-	curEvent := *hisEvent.ToCur()
-	curEvent.SetTagsMap()
-
-	// 绕过时间范围检查：设置时间范围为全量（0 到 int64 最大值），仅验证其他匹配条件（如标签、策略类型等）
-	f.AlertMute.MuteTimeType = models.TimeRange
-	f.AlertMute.Btime = 0             // 最小可能值（如 Unix 时间戳起点）
-	f.AlertMute.Etime = math.MaxInt64 // 最大可能值（int64 上限）
-
-	if !mute.MatchMute(&curEvent, &f.AlertMute) {
-		ginx.NewRender(c).Data("not match", nil)
-		return
-	}
-
-	ginx.NewRender(c).Data("mute test match", nil)
-
-}
-
 // Preview events (alert_cur_event) that match the mute strategy based on the following criteria:
 // business group ID (group_id, group_id), product (prod, rule_prod),
 // alert event severity (severities, severity), and event tags (tags, tags).

center/router/router_mw.go

@@ -9,7 +9,6 @@ import (
 	"strings"
 	"time"
 
-	"github.com/ccfos/nightingale/v6/center/cstats"
 	"github.com/ccfos/nightingale/v6/models"
 
 	"github.com/gin-gonic/gin"
@@ -388,17 +387,13 @@ func (rt *Router) createAuth(ctx context.Context, userIdentity string, td *Token
 	now := time.Now()
 
 	if err := rt.Redis.Set(ctx, rt.wrapJwtKey(td.AccessUuid), userIdentity, at.Sub(now)).Err(); err != nil {
-		cstats.RedisOperationLatency.WithLabelValues("set_token", "fail").Observe(time.Since(now).Seconds())
 		return err
 	}
 
 	if err := rt.Redis.Set(ctx, rt.wrapJwtKey(td.RefreshUuid), userIdentity, rte.Sub(now)).Err(); err != nil {
-		cstats.RedisOperationLatency.WithLabelValues("set_token", "fail").Observe(time.Since(now).Seconds())
 		return err
 	}
 
-	cstats.RedisOperationLatency.WithLabelValues("set_token", "success").Observe(time.Since(now).Seconds())
-
 	if rt.HTTP.JWTAuth.SingleLogin {
 		if err := rt.Redis.SAdd(ctx, rt.wrapJwtKey(username), rt.wrapJwtKey(td.AccessUuid), rt.wrapJwtKey(td.RefreshUuid)).Err(); err != nil {
 			return err
@@ -409,25 +404,11 @@ func (rt *Router) createAuth(ctx context.Context, userIdentity string, td *Token
 }
 
 func (rt *Router) fetchAuth(ctx context.Context, givenUuid string) (string, error) {
-	now := time.Now()
-	ret, err := rt.Redis.Get(ctx, rt.wrapJwtKey(givenUuid)).Result()
-	if err != nil {
-		cstats.RedisOperationLatency.WithLabelValues("get_token", "fail").Observe(time.Since(now).Seconds())
-	} else {
-		cstats.RedisOperationLatency.WithLabelValues("get_token", "success").Observe(time.Since(now).Seconds())
-	}
-
-	return ret, err
+	return rt.Redis.Get(ctx, rt.wrapJwtKey(givenUuid)).Result()
 }
 
 func (rt *Router) deleteAuth(ctx context.Context, givenUuid string) error {
-	err := rt.Redis.Del(ctx, rt.wrapJwtKey(givenUuid)).Err()
-	if err != nil {
-		cstats.RedisOperationLatency.WithLabelValues("del_token", "fail").Observe(time.Since(time.Now()).Seconds())
-	} else {
-		cstats.RedisOperationLatency.WithLabelValues("del_token", "success").Observe(time.Since(time.Now()).Seconds())
-	}
-	return err
+	return rt.Redis.Del(ctx, rt.wrapJwtKey(givenUuid)).Err()
 }
 
 func (rt *Router) deleteTokens(ctx context.Context, authD *AccessDetails) error {

center/router/router_notify_rule.go

@@ -31,7 +31,7 @@ func (rt *Router) notifyRulesAdd(c *gin.Context) {
 	for _, nr := range lst {
 		ginx.Dangerous(nr.Verify())
 		if !isAdmin && !slice.HaveIntersection(gids, nr.UserGroupIds) {
-			ginx.Bomb(http.StatusForbidden, "forbidden")
+			ginx.Bomb(http.StatusForbidden, "no permission")
 		}
 
 		nr.CreateBy = me.Username
@@ -56,8 +56,8 @@ func (rt *Router) notifyRulesDel(c *gin.Context) {
 		gids, err := models.MyGroupIds(rt.Ctx, me.Id)
 		ginx.Dangerous(err)
 		for _, t := range lst {
-			if !slice.HaveIntersection(gids, t.UserGroupIds) {
-				ginx.Bomb(http.StatusForbidden, "forbidden")
+			if !slice.HaveIntersection[int64](gids, t.UserGroupIds) {
+				ginx.Bomb(http.StatusForbidden, "no permission")
 			}
 		}
 	}
@@ -79,8 +79,8 @@ func (rt *Router) notifyRulePut(c *gin.Context) {
 	me := c.MustGet("user").(*models.User)
 	gids, err := models.MyGroupIds(rt.Ctx, me.Id)
 	ginx.Dangerous(err)
-	if !slice.HaveIntersection(gids, nr.UserGroupIds) && !me.IsAdmin() {
-		ginx.Bomb(http.StatusForbidden, "forbidden")
+	if !slice.HaveIntersection[int64](gids, nr.UserGroupIds) && !me.IsAdmin() {
+		ginx.Bomb(http.StatusForbidden, "no permission")
 	}
 
 	f.UpdateBy = me.Username
@@ -99,8 +99,8 @@ func (rt *Router) notifyRuleGet(c *gin.Context) {
 		ginx.Bomb(http.StatusNotFound, "notify rule not found")
 	}
 
-	if !slice.HaveIntersection(gids, nr.UserGroupIds) && !me.IsAdmin() {
-		ginx.Bomb(http.StatusForbidden, "forbidden")
+	if !slice.HaveIntersection[int64](gids, nr.UserGroupIds) && !me.IsAdmin() {
+		ginx.Bomb(http.StatusForbidden, "no permission")
 	}
 
 	ginx.NewRender(c).Data(nr, nil)
@@ -237,9 +237,6 @@ func (rt *Router) notifyTest(c *gin.Context) {
 		}
 
 	case "smtp":
-		if len(sendtos) == 0 {
-			ginx.Bomb(http.StatusBadRequest, "No valid email address in the user and team")
-		}
 		err := notifyChannel.SendEmailNow(events, tplContent, sendtos)
 		ginx.NewRender(c).Message(err)
 	case "script":

center/router/router_notify_tpl.go

@@ -45,7 +45,7 @@ func (rt *Router) notifyTplUpdateContent(c *gin.Context) {
 	ginx.Dangerous(err)
 
 	if notifyTpl.CreateBy != user.Username && !user.IsAdmin() {
-		ginx.Bomb(403, "forbidden")
+		ginx.Bomb(403, "no permission")
 	}
 
 	f.UpdateAt = time.Now().Unix()
@@ -64,7 +64,7 @@ func (rt *Router) notifyTplUpdate(c *gin.Context) {
 	ginx.Dangerous(err)
 
 	if notifyTpl.CreateBy != user.Username && !user.IsAdmin() {
-		ginx.Bomb(403, "forbidden")
+		ginx.Bomb(403, "no permission")
 	}
 
 	// get the count of the same channel and name but different id
@@ -188,7 +188,7 @@ func (rt *Router) notifyTplDel(c *gin.Context) {
 	ginx.Dangerous(err)
 
 	if notifyTpl.CreateBy != user.Username && !user.IsAdmin() {
-		ginx.Bomb(403, "forbidden")
+		ginx.Bomb(403, "no permission")
 	}
 
 	ginx.NewRender(c).Message(f.NotifyTplDelete(rt.Ctx, id))

center/router/router_query.go

@@ -47,7 +47,7 @@ func QueryLogBatchConcurrently(anonymousAccess bool, ctx *gin.Context, f QueryFr
 
 	for _, q := range f.Queries {
 		if !anonymousAccess && !CheckDsPerm(ctx, q.Did, q.DsCate, q) {
-			return LogResp{}, fmt.Errorf("forbidden")
+			return LogResp{}, fmt.Errorf("no permission")
 		}
 
 		plug, exists := dscache.DsCache.Get(q.DsCate, q.Did)
@@ -114,7 +114,7 @@ func QueryDataConcurrently(anonymousAccess bool, ctx *gin.Context, f models.Quer
 
 	for _, q := range f.Querys {
 		if !anonymousAccess && !CheckDsPerm(ctx, f.DatasourceId, f.Cate, q) {
-			return nil, fmt.Errorf("forbidden")
+			return nil, fmt.Errorf("no permission")
 		}
 
 		plug, exists := dscache.DsCache.Get(f.Cate, f.DatasourceId)
@@ -185,7 +185,7 @@ func QueryLogConcurrently(anonymousAccess bool, ctx *gin.Context, f models.Query
 
 	for _, q := range f.Querys {
 		if !anonymousAccess && !CheckDsPerm(ctx, f.DatasourceId, f.Cate, q) {
-			return LogResp{}, fmt.Errorf("forbidden")
+			return LogResp{}, fmt.Errorf("no permission")
 		}
 
 		plug, exists := dscache.DsCache.Get(f.Cate, f.DatasourceId)
@@ -244,7 +244,7 @@ func (rt *Router) QueryLog(c *gin.Context) {
 	var resp []interface{}
 	for _, q := range f.Querys {
 		if !rt.Center.AnonymousAccess.PromQuerier && !CheckDsPerm(c, f.DatasourceId, f.Cate, q) {
-			ginx.Bomb(200, "forbidden")
+			ginx.Bomb(200, "no permission")
 		}
 
 		plug, exists := dscache.DsCache.Get("elasticsearch", f.DatasourceId)

center/router/router_source_token.go

@@ -1,36 +0,0 @@
-package router
-
-import (
-	"net/http"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/google/uuid"
-
-	"github.com/gin-gonic/gin"
-	"github.com/toolkits/pkg/ginx"
-)
-
-// sourceTokenAdd 生成新的源令牌
-func (rt *Router) sourceTokenAdd(c *gin.Context) {
-	var f models.SourceToken
-	ginx.BindJSON(c, &f)
-
-	if f.ExpireAt > 0 && f.ExpireAt <= time.Now().Unix() {
-		ginx.Bomb(http.StatusBadRequest, "expire time must be in the future")
-	}
-
-	token := uuid.New().String()
-
-	username := c.MustGet("username").(string)
-
-	f.Token = token
-	f.CreateBy = username
-	f.CreateAt = time.Now().Unix()
-
-	err := f.Add(rt.Ctx)
-	ginx.Dangerous(err)
-
-	go models.CleanupExpiredTokens(rt.Ctx)
-	ginx.NewRender(c).Data(token, nil)
-}

center/router/router_target.go

@@ -57,11 +57,8 @@ func (rt *Router) targetGets(c *gin.Context) {
 
 	var err error
 	if len(bgids) > 0 {
-		// 如果用户当前查看的是未归组机器，会传入 bgids = [0]，此时是不需要校验的，故而排除这种情况
-		if !(len(bgids) == 1 && bgids[0] == 0) {
-			for _, gid := range bgids {
-				rt.bgroCheck(c, gid)
-			}
+		for _, gid := range bgids {
+			rt.bgroCheck(c, gid)
 		}
 	} else {
 		user := c.MustGet("user").(*models.User)
@@ -461,7 +458,7 @@ func (rt *Router) targetBindBgids(c *gin.Context) {
 			ginx.Dangerous(err)
 
 			if !can {
-				ginx.Bomb(http.StatusForbidden, "forbidden")
+				ginx.Bomb(http.StatusForbidden, "No permission. You are not admin of BG(%s)", bg.Name)
 			}
 		}
 		isNeverGrouped, checkErr := haveNeverGroupedIdent(rt.Ctx, f.Idents)
@@ -471,7 +468,7 @@ func (rt *Router) targetBindBgids(c *gin.Context) {
 			can, err := user.CheckPerm(rt.Ctx, "/targets/bind")
 			ginx.Dangerous(err)
 			if !can {
-				ginx.Bomb(http.StatusForbidden, "forbidden")
+				ginx.Bomb(http.StatusForbidden, "No permission. Only admin can assign BG")
 			}
 		}
 	}
@@ -556,7 +553,7 @@ func (rt *Router) checkTargetPerm(c *gin.Context, idents []string) {
 	ginx.Dangerous(err)
 
 	if len(nopri) > 0 {
-		ginx.Bomb(http.StatusForbidden, "forbidden")
+		ginx.Bomb(http.StatusForbidden, "No permission to operate the targets: %s", strings.Join(nopri, ", "))
 	}
 }
 
@@ -579,11 +576,11 @@ func (rt *Router) targetsOfAlertRule(c *gin.Context) {
 }
 
 func (rt *Router) checkTargetsExistByIndent(idents []string) {
-	notExists, err := models.TargetNoExistIdents(rt.Ctx, idents)
+	existingIdents, err := models.TargetNoExistIdents(rt.Ctx, idents)
 	ginx.Dangerous(err)
 
-	if len(notExists) > 0 {
-		ginx.Bomb(http.StatusBadRequest, "targets not exist: %s", strings.Join(notExists, ", "))
+	if len(existingIdents) > 0 {
+		ginx.Bomb(http.StatusBadRequest, "targets not exist: %s", strings.Join(existingIdents, ","))
 	}
 }
 

center/router/router_task.go

@@ -1,7 +1,6 @@
 package router
 
 import (
-	"strings"
 	"time"
 
 	"github.com/ccfos/nightingale/v6/alert/sender"
@@ -85,6 +84,20 @@ func (rt *Router) taskGetsByGids(c *gin.Context) {
 	}, nil)
 }
 
+type taskForm struct {
+	Title     string   `json:"title" binding:"required"`
+	Account   string   `json:"account" binding:"required"`
+	Batch     int      `json:"batch"`
+	Tolerance int      `json:"tolerance"`
+	Timeout   int      `json:"timeout"`
+	Pause     string   `json:"pause"`
+	Script    string   `json:"script" binding:"required"`
+	Args      string   `json:"args"`
+	Action    string   `json:"action" binding:"required"`
+	Creator   string   `json:"creator"`
+	Hosts     []string `json:"hosts" binding:"required"`
+}
+
 func (rt *Router) taskRecordAdd(c *gin.Context) {
 	var f *models.TaskRecord
 	ginx.BindJSON(c, &f)
@@ -99,14 +112,6 @@ func (rt *Router) taskAdd(c *gin.Context) {
 
 	var f models.TaskForm
 	ginx.BindJSON(c, &f)
-	// 把 f.Hosts 中的空字符串过滤掉
-	hosts := make([]string, 0, len(f.Hosts))
-	for i := range f.Hosts {
-		if strings.TrimSpace(f.Hosts[i]) != "" {
-			hosts = append(hosts, strings.TrimSpace(f.Hosts[i]))
-		}
-	}
-	f.Hosts = hosts
 
 	bgid := ginx.UrlParamInt64(c, "id")
 	user := c.MustGet("user").(*models.User)

center/router/router_task_tpl.go

@@ -119,18 +119,6 @@ type taskTplForm struct {
 	Hosts     []string `json:"hosts"`
 }
 
-func (f *taskTplForm) Verify() {
-	// 传入的 f.Hosts 可能是 []string{"", "a", "b"}，需要过滤掉空字符串
-	args := make([]string, 0, len(f.Hosts))
-	for _, ident := range f.Hosts {
-		if strings.TrimSpace(ident) != "" {
-			args = append(args, strings.TrimSpace(ident))
-		}
-	}
-
-	f.Hosts = args
-}
-
 func (rt *Router) taskTplAdd(c *gin.Context) {
 	if !rt.Ibex.Enable {
 		ginx.Bomb(400, i18n.Sprintf(c.GetHeader("X-Language"), "This functionality has not been enabled. Please contact the system administrator to activate it."))
@@ -139,7 +127,6 @@ func (rt *Router) taskTplAdd(c *gin.Context) {
 
 	var f taskTplForm
 	ginx.BindJSON(c, &f)
-	f.Verify()
 
 	user := c.MustGet("user").(*models.User)
 	now := time.Now().Unix()
@@ -183,7 +170,6 @@ func (rt *Router) taskTplPut(c *gin.Context) {
 
 	var f taskTplForm
 	ginx.BindJSON(c, &f)
-	f.Verify()
 
 	rt.checkTargetsExistByIndent(f.Hosts)
 

center/router/router_user_variable_config.go

@@ -40,7 +40,7 @@ func (rt *Router) userVariableConfigPut(context *gin.Context) {
 	user := context.MustGet("user").(*models.User)
 	if !user.IsAdmin() && f.CreateBy != user.Username {
 		// only admin or creator can update
-		ginx.Bomb(403, "forbidden")
+		ginx.Bomb(403, "no permission")
 	}
 
 	ginx.NewRender(context).Message(models.ConfigsUserVariableUpdate(rt.Ctx, f))
@@ -54,7 +54,7 @@ func (rt *Router) userVariableConfigDel(context *gin.Context) {
 	user := context.MustGet("user").(*models.User)
 	if !user.IsAdmin() && configs.CreateBy != user.Username {
 		// only admin or creator can delete
-		ginx.Bomb(403, "forbidden")
+		ginx.Bomb(403, "no permission")
 	}
 
 	if configs != nil && configs.External == models.ConfigExternal {

cmd/edge/edge.go

@@ -54,7 +54,7 @@ func Initialize(configDir string, cryptoKey string) (func(), error) {
 	targetCache := memsto.NewTargetCache(ctx, syncStats, redis)
 	busiGroupCache := memsto.NewBusiGroupCache(ctx, syncStats)
 	configCvalCache := memsto.NewCvalCache(ctx, syncStats)
-	idents := idents.New(ctx, redis, config.Pushgw)
+	idents := idents.New(ctx, redis)
 	metas := metas.New(redis)
 	writers := writer.NewWriters(config.Pushgw)
 	pushgwRouter := pushgwrt.New(config.HTTP, config.Pushgw, config.Alert, targetCache, busiGroupCache, idents, metas, writers, ctx)

datasource/datasource.go

@@ -53,20 +53,6 @@ func init() {
 		PluginType:     "ck",
 		PluginTypeName: "ClickHouse",
 	}
-
-	DatasourceTypes[5] = DatasourceType{
-		Id:             5,
-		Category:       "timeseries",
-		PluginType:     "mysql",
-		PluginTypeName: "MySQL",
-	}
-
-	DatasourceTypes[6] = DatasourceType{
-		Id:             6,
-		Category:       "timeseries",
-		PluginType:     "pgsql",
-		PluginTypeName: "PostgreSQL",
-	}
 }
 
 type NewDatasrouceFn func(settings map[string]interface{}) (Datasource, error)

datasource/mysql/mysql.go

@@ -1,227 +0,0 @@
-package mysql
-
-import (
-	"context"
-	"fmt"
-	"strings"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/datasource"
-	"github.com/ccfos/nightingale/v6/dskit/mysql"
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/macros"
-
-	"github.com/mitchellh/mapstructure"
-	"github.com/toolkits/pkg/logger"
-)
-
-const (
-	MySQLType = "mysql"
-)
-
-func init() {
-	datasource.RegisterDatasource(MySQLType, new(MySQL))
-}
-
-type MySQL struct {
-	mysql.MySQL `json:",inline" mapstructure:",squash"`
-}
-
-type QueryParam struct {
-	Ref      string          `json:"ref" mapstructure:"ref"`
-	Database string          `json:"database" mapstructure:"database"`
-	Table    string          `json:"table" mapstructure:"table"`
-	SQL      string          `json:"sql" mapstructure:"sql"`
-	Keys     datasource.Keys `json:"keys" mapstructure:"keys"`
-	From     int64           `json:"from" mapstructure:"from"`
-	To       int64           `json:"to" mapstructure:"to"`
-}
-
-func (m *MySQL) InitClient() error {
-	if len(m.Shards) == 0 {
-		return fmt.Errorf("not found mysql addr, please check datasource config")
-	}
-	if _, err := m.NewConn(context.TODO(), ""); err != nil {
-		return err
-	}
-	return nil
-}
-
-func (m *MySQL) Init(settings map[string]interface{}) (datasource.Datasource, error) {
-	newest := new(MySQL)
-	err := mapstructure.Decode(settings, newest)
-	return newest, err
-}
-
-func (m *MySQL) Validate(ctx context.Context) error {
-	if len(m.Shards) == 0 || len(strings.TrimSpace(m.Shards[0].Addr)) == 0 {
-		return fmt.Errorf("mysql addr is invalid, please check datasource setting")
-	}
-
-	if len(strings.TrimSpace(m.Shards[0].User)) == 0 {
-		return fmt.Errorf("mysql user is invalid, please check datasource setting")
-	}
-
-	return nil
-}
-
-// Equal compares whether two objects are the same, used for caching
-func (m *MySQL) Equal(p datasource.Datasource) bool {
-	newest, ok := p.(*MySQL)
-	if !ok {
-		logger.Errorf("unexpected plugin type, expected is mysql")
-		return false
-	}
-
-	if len(m.Shards) == 0 || len(newest.Shards) == 0 {
-		return false
-	}
-
-	oldShard := m.Shards[0]
-	newShard := newest.Shards[0]
-
-	if oldShard.Addr != newShard.Addr {
-		return false
-	}
-
-	if oldShard.User != newShard.User {
-		return false
-	}
-
-	if oldShard.Password != newShard.Password {
-		return false
-	}
-
-	if oldShard.MaxQueryRows != newShard.MaxQueryRows {
-		return false
-	}
-
-	if oldShard.Timeout != newShard.Timeout {
-		return false
-	}
-
-	if oldShard.MaxIdleConns != newShard.MaxIdleConns {
-		return false
-	}
-
-	if oldShard.MaxOpenConns != newShard.MaxOpenConns {
-		return false
-	}
-
-	if oldShard.ConnMaxLifetime != newShard.ConnMaxLifetime {
-		return false
-	}
-
-	return true
-}
-
-func (m *MySQL) MakeLogQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
-	return nil, nil
-}
-
-func (m *MySQL) MakeTSQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
-	return nil, nil
-}
-
-func (m *MySQL) QueryMapData(ctx context.Context, query interface{}) ([]map[string]string, error) {
-	return nil, nil
-}
-
-func (m *MySQL) QueryData(ctx context.Context, query interface{}) ([]models.DataResp, error) {
-	mysqlQueryParam := new(QueryParam)
-	if err := mapstructure.Decode(query, mysqlQueryParam); err != nil {
-		return nil, err
-	}
-
-	if strings.Contains(mysqlQueryParam.SQL, "$__") {
-		var err error
-		mysqlQueryParam.SQL, err = macros.Macro(mysqlQueryParam.SQL, mysqlQueryParam.From, mysqlQueryParam.To)
-		if err != nil {
-			return nil, err
-		}
-	}
-
-	if mysqlQueryParam.Keys.ValueKey == "" {
-		return nil, fmt.Errorf("valueKey is required")
-	}
-
-	timeout := m.Shards[0].Timeout
-	if timeout == 0 {
-		timeout = 60
-	}
-
-	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
-	defer cancel()
-
-	items, err := m.QueryTimeseries(timeoutCtx, &sqlbase.QueryParam{
-		Sql: mysqlQueryParam.SQL,
-		Keys: types.Keys{
-			ValueKey: mysqlQueryParam.Keys.ValueKey,
-			LabelKey: mysqlQueryParam.Keys.LabelKey,
-			TimeKey:  mysqlQueryParam.Keys.TimeKey,
-		},
-	})
-
-	if err != nil {
-		logger.Warningf("query:%+v get data err:%v", mysqlQueryParam, err)
-		return []models.DataResp{}, err
-	}
-	data := make([]models.DataResp, 0)
-	for i := range items {
-		data = append(data, models.DataResp{
-			Ref:    mysqlQueryParam.Ref,
-			Metric: items[i].Metric,
-			Values: items[i].Values,
-		})
-	}
-
-	return data, nil
-}
-
-func (m *MySQL) QueryLog(ctx context.Context, query interface{}) ([]interface{}, int64, error) {
-	mysqlQueryParam := new(QueryParam)
-	if err := mapstructure.Decode(query, mysqlQueryParam); err != nil {
-		return nil, 0, err
-	}
-
-	if strings.Contains(mysqlQueryParam.SQL, "$__") {
-		var err error
-		mysqlQueryParam.SQL, err = macros.Macro(mysqlQueryParam.SQL, mysqlQueryParam.From, mysqlQueryParam.To)
-		if err != nil {
-			return nil, 0, err
-		}
-	}
-
-	timeout := m.Shards[0].Timeout
-	if timeout == 0 {
-		timeout = 60
-	}
-
-	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
-	defer cancel()
-
-	items, err := m.Query(timeoutCtx, &sqlbase.QueryParam{
-		Sql: mysqlQueryParam.SQL,
-	})
-
-	if err != nil {
-		logger.Warningf("query:%+v get data err:%v", mysqlQueryParam, err)
-		return []interface{}{}, 0, err
-	}
-	logs := make([]interface{}, 0)
-	for i := range items {
-		logs = append(logs, items[i])
-	}
-
-	return logs, 0, nil
-}
-
-func (m *MySQL) DescribeTable(ctx context.Context, query interface{}) ([]*types.ColumnProperty, error) {
-	mysqlQueryParam := new(QueryParam)
-	if err := mapstructure.Decode(query, mysqlQueryParam); err != nil {
-		return nil, err
-	}
-	return m.DescTable(ctx, mysqlQueryParam.Database, mysqlQueryParam.Table)
-}

datasource/postgresql/postgresql.go

@@ -1,346 +0,0 @@
-package postgresql
-
-import (
-	"context"
-	"fmt"
-	"regexp"
-	"strings"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/datasource"
-	"github.com/ccfos/nightingale/v6/pkg/macros"
-
-	"github.com/ccfos/nightingale/v6/dskit/postgres"
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/mitchellh/mapstructure"
-	"github.com/toolkits/pkg/logger"
-)
-
-const (
-	PostgreSQLType = "pgsql"
-)
-
-var (
-	regx = "(?i)from\\s+([a-zA-Z0-9_]+)\\.([a-zA-Z0-9_]+)\\.([a-zA-Z0-9_]+)"
-)
-
-func init() {
-	datasource.RegisterDatasource(PostgreSQLType, new(PostgreSQL))
-}
-
-type PostgreSQL struct {
-	Shards []*postgres.PostgreSQL `json:"pgsql.shards" mapstructure:"pgsql.shards"`
-}
-
-type QueryParam struct {
-	Ref      string          `json:"ref" mapstructure:"ref"`
-	Database string          `json:"database" mapstructure:"database"`
-	Table    string          `json:"table" mapstructure:"table"`
-	SQL      string          `json:"sql" mapstructure:"sql"`
-	Keys     datasource.Keys `json:"keys" mapstructure:"keys"`
-	From     int64           `json:"from" mapstructure:"from"`
-	To       int64           `json:"to" mapstructure:"to"`
-}
-
-func (p *PostgreSQL) InitClient() error {
-	if len(p.Shards) == 0 {
-		return fmt.Errorf("not found postgresql addr, please check datasource config")
-	}
-	for _, shard := range p.Shards {
-		if db, err := shard.NewConn(context.TODO(), "postgres"); err != nil {
-			defer sqlbase.CloseDB(db)
-			return err
-		}
-	}
-	return nil
-}
-
-func (p *PostgreSQL) Init(settings map[string]interface{}) (datasource.Datasource, error) {
-	newest := new(PostgreSQL)
-	err := mapstructure.Decode(settings, newest)
-	return newest, err
-}
-
-func (p *PostgreSQL) Validate(ctx context.Context) error {
-	if len(p.Shards) == 0 || len(strings.TrimSpace(p.Shards[0].Addr)) == 0 {
-		return fmt.Errorf("postgresql addr is invalid, please check datasource setting")
-	}
-
-	if len(strings.TrimSpace(p.Shards[0].User)) == 0 {
-		return fmt.Errorf("postgresql user is invalid, please check datasource setting")
-	}
-
-	return nil
-}
-
-// Equal compares whether two objects are the same, used for caching
-func (p *PostgreSQL) Equal(d datasource.Datasource) bool {
-	newest, ok := d.(*PostgreSQL)
-	if !ok {
-		logger.Errorf("unexpected plugin type, expected is postgresql")
-		return false
-	}
-
-	if len(p.Shards) == 0 || len(newest.Shards) == 0 {
-		return false
-	}
-
-	oldShard := p.Shards[0]
-	newShard := newest.Shards[0]
-
-	if oldShard.Addr != newShard.Addr {
-		return false
-	}
-
-	if oldShard.User != newShard.User {
-		return false
-	}
-
-	if oldShard.Password != newShard.Password {
-		return false
-	}
-
-	if oldShard.MaxQueryRows != newShard.MaxQueryRows {
-		return false
-	}
-
-	if oldShard.Timeout != newShard.Timeout {
-		return false
-	}
-
-	if oldShard.MaxIdleConns != newShard.MaxIdleConns {
-		return false
-	}
-
-	if oldShard.MaxOpenConns != newShard.MaxOpenConns {
-		return false
-	}
-
-	if oldShard.ConnMaxLifetime != newShard.ConnMaxLifetime {
-		return false
-	}
-
-	return true
-}
-
-func (p *PostgreSQL) ShowDatabases(ctx context.Context) ([]string, error) {
-	return p.Shards[0].ShowDatabases(ctx, "")
-}
-
-func (p *PostgreSQL) ShowTables(ctx context.Context, database string) ([]string, error) {
-	p.Shards[0].DB = database
-	rets, err := p.Shards[0].ShowTables(ctx, "")
-	if err != nil {
-		return nil, err
-	}
-	tables := make([]string, 0, len(rets))
-	for scheme, tabs := range rets {
-		for _, tab := range tabs {
-			tables = append(tables, scheme+"."+tab)
-		}
-	}
-	return tables, nil
-}
-
-func (p *PostgreSQL) MakeLogQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
-	return nil, nil
-}
-
-func (p *PostgreSQL) MakeTSQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
-	return nil, nil
-}
-
-func (p *PostgreSQL) QueryMapData(ctx context.Context, query interface{}) ([]map[string]string, error) {
-	return nil, nil
-}
-
-func (p *PostgreSQL) QueryData(ctx context.Context, query interface{}) ([]models.DataResp, error) {
-	postgresqlQueryParam := new(QueryParam)
-	if err := mapstructure.Decode(query, postgresqlQueryParam); err != nil {
-		return nil, err
-	}
-
-	if strings.Contains(postgresqlQueryParam.SQL, "$__") {
-		var err error
-		postgresqlQueryParam.SQL, err = macros.Macro(postgresqlQueryParam.SQL, postgresqlQueryParam.From, postgresqlQueryParam.To)
-		if err != nil {
-			return nil, err
-		}
-	}
-	if postgresqlQueryParam.Database != "" {
-		p.Shards[0].DB = postgresqlQueryParam.Database
-	} else {
-		db, err := parseDBName(postgresqlQueryParam.SQL)
-		if err != nil {
-			return nil, err
-		}
-		p.Shards[0].DB = db
-	}
-
-	timeout := p.Shards[0].Timeout
-	if timeout == 0 {
-		timeout = 60
-	}
-	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
-	defer cancel()
-
-	items, err := p.Shards[0].QueryTimeseries(timeoutCtx, &sqlbase.QueryParam{
-		Sql: postgresqlQueryParam.SQL,
-		Keys: types.Keys{
-			ValueKey: postgresqlQueryParam.Keys.ValueKey,
-			LabelKey: postgresqlQueryParam.Keys.LabelKey,
-			TimeKey:  postgresqlQueryParam.Keys.TimeKey,
-		},
-	})
-
-	if err != nil {
-		logger.Warningf("query:%+v get data err:%v", postgresqlQueryParam, err)
-		return []models.DataResp{}, err
-	}
-	data := make([]models.DataResp, 0)
-	for i := range items {
-		data = append(data, models.DataResp{
-			Ref:    postgresqlQueryParam.Ref,
-			Metric: items[i].Metric,
-			Values: items[i].Values,
-		})
-	}
-
-	// parse resp to time series data
-	logger.Infof("req:%+v keys:%+v \n data:%v", postgresqlQueryParam, postgresqlQueryParam.Keys, data)
-
-	return data, nil
-}
-
-func (p *PostgreSQL) QueryLog(ctx context.Context, query interface{}) ([]interface{}, int64, error) {
-	postgresqlQueryParam := new(QueryParam)
-	if err := mapstructure.Decode(query, postgresqlQueryParam); err != nil {
-		return nil, 0, err
-	}
-	if postgresqlQueryParam.Database != "" {
-		p.Shards[0].DB = postgresqlQueryParam.Database
-	} else {
-		db, err := parseDBName(postgresqlQueryParam.SQL)
-		if err != nil {
-			return nil, 0, err
-		}
-		p.Shards[0].DB = db
-	}
-
-	if strings.Contains(postgresqlQueryParam.SQL, "$__") {
-		var err error
-		postgresqlQueryParam.SQL, err = macros.Macro(postgresqlQueryParam.SQL, postgresqlQueryParam.From, postgresqlQueryParam.To)
-		if err != nil {
-			return nil, 0, err
-		}
-	}
-
-	timeout := p.Shards[0].Timeout
-	if timeout == 0 {
-		timeout = 60
-	}
-	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
-	defer cancel()
-	items, err := p.Shards[0].Query(timeoutCtx, &sqlbase.QueryParam{
-		Sql: postgresqlQueryParam.SQL,
-	})
-	if err != nil {
-		logger.Warningf("query:%+v get data err:%v", postgresqlQueryParam, err)
-		return []interface{}{}, 0, err
-	}
-	logs := make([]interface{}, 0)
-	for i := range items {
-		logs = append(logs, items[i])
-	}
-
-	return logs, 0, nil
-}
-
-func (p *PostgreSQL) DescribeTable(ctx context.Context, query interface{}) ([]*types.ColumnProperty, error) {
-	postgresqlQueryParam := new(QueryParam)
-	if err := mapstructure.Decode(query, postgresqlQueryParam); err != nil {
-		return nil, err
-	}
-	p.Shards[0].DB = postgresqlQueryParam.Database
-	pairs := strings.Split(postgresqlQueryParam.Table, ".") // format: scheme.table_name
-	scheme := ""
-	table := postgresqlQueryParam.Table
-	if len(pairs) == 2 {
-		scheme = pairs[0]
-		table = pairs[1]
-	}
-	return p.Shards[0].DescTable(ctx, scheme, table)
-}
-
-func parseDBName(sql string) (db string, err error) {
-	re := regexp.MustCompile(regx)
-	matches := re.FindStringSubmatch(sql)
-	if len(matches) != 4 {
-		return "", fmt.Errorf("no valid table name in format database.schema.table found")
-	}
-	return matches[1], nil
-}
-
-func extractColumns(sql string) ([]string, error) {
-	// 将 SQL 转换为小写以简化匹配
-	sql = strings.ToLower(sql)
-
-	// 匹配 SELECT 和 FROM 之间的内容
-	re := regexp.MustCompile(`select\s+(.*?)\s+from`)
-	matches := re.FindStringSubmatch(sql)
-
-	if len(matches) < 2 {
-		return nil, fmt.Errorf("no columns found or invalid SQL syntax")
-	}
-
-	// 提取列部分
-	columnsString := matches[1]
-
-	// 分割列
-	columns := splitColumns(columnsString)
-
-	// 清理每个列名
-	for i, col := range columns {
-		columns[i] = strings.TrimSpace(col)
-	}
-
-	return columns, nil
-}
-
-func splitColumns(columnsString string) []string {
-	var columns []string
-	var currentColumn strings.Builder
-	parenthesesCount := 0
-	inQuotes := false
-
-	for _, char := range columnsString {
-		switch char {
-		case '(':
-			parenthesesCount++
-			currentColumn.WriteRune(char)
-		case ')':
-			parenthesesCount--
-			currentColumn.WriteRune(char)
-		case '\'', '"':
-			inQuotes = !inQuotes
-			currentColumn.WriteRune(char)
-		case ',':
-			if parenthesesCount == 0 && !inQuotes {
-				columns = append(columns, currentColumn.String())
-				currentColumn.Reset()
-			} else {
-				currentColumn.WriteRune(char)
-			}
-		default:
-			currentColumn.WriteRune(char)
-		}
-	}
-
-	if currentColumn.Len() > 0 {
-		columns = append(columns, currentColumn.String())
-	}
-
-	return columns
-}

docker/compose-postgres/initsql_for_postgres/a-n9e-for-Postgres.sql

@@ -204,7 +204,6 @@ CREATE TABLE board (
     public smallint not null default 0 ,
     built_in smallint not null default 0 ,
     hide smallint not null default 0 ,
-    public_cate bigint NOT NULL DEFAULT 0,
     create_at bigint not null default 0,
     create_by varchar(64) not null default '',
     update_at bigint not null default 0,
@@ -218,7 +217,6 @@ COMMENT ON COLUMN board.tags IS 'split by space';
 COMMENT ON COLUMN board.public IS '0:false 1:true';
 COMMENT ON COLUMN board.built_in IS '0:false 1:true';
 COMMENT ON COLUMN board.hide IS '0:false 1:true';
-COMMENT ON COLUMN board.public_cate IS '0 anonymous 1 login 2 busi';
 
 
 -- for dashboard new version
@@ -431,31 +429,43 @@ CREATE TABLE target (
     ident varchar(191) not null,
     note varchar(255) not null default '',
     tags varchar(512) not null default '',
-    host_tags text,
     host_ip varchar(15) default '',
     agent_version varchar(255) default '',
     engine_name varchar(255) default '',
-    os varchar(31) default '',
     update_at bigint not null default 0,
     PRIMARY KEY (id),
     UNIQUE (ident)
 );
 
 CREATE INDEX ON target (group_id);
-CREATE INDEX idx_host_ip ON target (host_ip);
-CREATE INDEX idx_agent_version ON target (agent_version);
-CREATE INDEX idx_engine_name ON target (engine_name);
-CREATE INDEX idx_os ON target (os);
 
 COMMENT ON COLUMN target.group_id IS 'busi group id';
 COMMENT ON COLUMN target.ident IS 'target id';
 COMMENT ON COLUMN target.note IS 'append to alert event as field';
 COMMENT ON COLUMN target.tags IS 'append to series data as tags, split by space, append external space at suffix';
-COMMENT ON COLUMN target.host_tags IS 'global labels set in conf file';
 COMMENT ON COLUMN target.host_ip IS 'IPv4 string';
 COMMENT ON COLUMN target.agent_version IS 'agent version';
 COMMENT ON COLUMN target.engine_name IS 'engine_name';
-COMMENT ON COLUMN target.os IS 'os type';
+-- case1: target_idents; case2: target_tags
+-- CREATE TABLE collect_rule (
+--     id bigserial,
+--     group_id bigint not null default 0 comment 'busi group id',
+--     cluster varchar(128) not null,
+--     target_idents varchar(512) not null default '' comment 'ident list, split by space',
+--     target_tags varchar(512) not null default '' comment 'filter targets by tags, split by space',
+--     name varchar(191) not null default '',
+--     note varchar(255) not null default '',
+--     step int not null,
+--     type varchar(64) not null comment 'e.g. port proc log plugin',
+--     data text not null,
+--     append_tags varchar(255) not null default '' comment 'split by space: e.g. mod=n9e dept=cloud',
+--     create_at bigint not null default 0,
+--     create_by varchar(64) not null default '',
+--     update_at bigint not null default 0,
+--     update_by varchar(64) not null default '',
+--     PRIMARY KEY (id),
+--     KEY (group_id, type, name)
+-- ) ;
 
 CREATE TABLE metric_view (
     id bigserial,
@@ -724,7 +734,6 @@ CREATE TABLE datasource
 (
     id serial,
     name varchar(191) not null default '',
-    identifier varchar(255) not null default '',
     description varchar(255) not null default '',
     category varchar(255) not null default '',
     plugin_id int  not null default 0,
@@ -742,8 +751,8 @@ CREATE TABLE datasource
     updated_by varchar(64) not null default '',
     UNIQUE (name),
     PRIMARY KEY (id)
-) ;
-
+) ; 
+  
 CREATE TABLE builtin_cate (
     id bigserial,
     name varchar(191) not null,
@@ -786,12 +795,10 @@ CREATE TABLE es_index_pattern (
     create_by varchar(64) default '',
     update_at bigint default '0',
     update_by varchar(64) default '',
-    note varchar(4096) not null default '',
     PRIMARY KEY (id),
     UNIQUE (datasource_id, name)
 ) ;
 COMMENT ON COLUMN es_index_pattern.datasource_id IS 'datasource id';
-COMMENT ON COLUMN es_index_pattern.note IS 'description of metric in Chinese';
 
 CREATE TABLE builtin_metrics (
   id bigserial,
@@ -806,7 +813,6 @@ CREATE TABLE builtin_metrics (
   created_by varchar(191) NOT NULL DEFAULT '',
   updated_at bigint NOT NULL DEFAULT 0,
   updated_by varchar(191) NOT NULL DEFAULT '',
-  uuid BIGINT NOT NULL DEFAULT 0,
   PRIMARY KEY (id),
   UNIQUE (lang, collector, typ, name)
 );
@@ -828,7 +834,6 @@ COMMENT ON COLUMN builtin_metrics.created_at IS 'create time';
 COMMENT ON COLUMN builtin_metrics.created_by IS 'creator';
 COMMENT ON COLUMN builtin_metrics.updated_at IS 'update time';
 COMMENT ON COLUMN builtin_metrics.updated_by IS 'updater';
-COMMENT ON COLUMN builtin_metrics.uuid IS 'unique identifier';
 
 CREATE TABLE metric_filter (
   id BIGSERIAL PRIMARY KEY,
@@ -898,128 +903,4 @@ CREATE TABLE dash_annotation (
     create_by varchar(64) not null default '',
     update_at bigint not null default 0,
     update_by varchar(64) not null default ''
-);
-
-CREATE TABLE source_token (
-    id bigserial PRIMARY KEY,
-    source_type varchar(64) NOT NULL DEFAULT '',
-    source_id varchar(255) NOT NULL DEFAULT '',
-    token varchar(255) NOT NULL DEFAULT '',
-    expire_at bigint NOT NULL DEFAULT 0,
-    create_at bigint NOT NULL DEFAULT 0,
-    create_by varchar(64) NOT NULL DEFAULT ''
-);
-
-CREATE INDEX idx_source_token_type_id_token ON source_token (source_type, source_id, token);
-
-CREATE TABLE notification_record (
-    id BIGSERIAL PRIMARY KEY,
-    notify_rule_id BIGINT NOT NULL DEFAULT 0,
-    event_id bigint NOT NULL,
-    sub_id bigint DEFAULT NULL,
-    channel varchar(255) NOT NULL,
-    status bigint DEFAULT NULL,
-    target varchar(1024) NOT NULL,
-    details varchar(2048) DEFAULT '',
-    created_at bigint NOT NULL
-);
-
-CREATE INDEX idx_evt ON notification_record (event_id);
-
-COMMENT ON COLUMN notification_record.event_id IS 'event history id';
-COMMENT ON COLUMN notification_record.sub_id IS 'subscribed rule id';
-COMMENT ON COLUMN notification_record.channel IS 'notification channel name';
-COMMENT ON COLUMN notification_record.status IS 'notification status';
-COMMENT ON COLUMN notification_record.target IS 'notification target';
-COMMENT ON COLUMN notification_record.details IS 'notification other info';
-COMMENT ON COLUMN notification_record.created_at IS 'create time';
-
-CREATE TABLE target_busi_group (
-    id BIGSERIAL PRIMARY KEY,
-    target_ident varchar(191) NOT NULL,
-    group_id bigint NOT NULL,
-    update_at bigint NOT NULL
-);
-
-CREATE UNIQUE INDEX idx_target_group ON target_busi_group (target_ident, group_id);
-
-CREATE TABLE user_token (
-    id BIGSERIAL PRIMARY KEY,
-    username varchar(255) NOT NULL DEFAULT '',
-    token_name varchar(255) NOT NULL DEFAULT '',
-    token varchar(255) NOT NULL DEFAULT '',
-    create_at bigint NOT NULL DEFAULT 0,
-    last_used bigint NOT NULL DEFAULT 0
-);
-
-CREATE TABLE notify_rule (
-    id bigserial PRIMARY KEY,
-    name varchar(255) NOT NULL,
-    description text,
-    enable smallint NOT NULL DEFAULT 0,
-    user_group_ids varchar(255) NOT NULL DEFAULT '',
-    notify_configs text,
-    pipeline_configs text,
-    create_at bigint NOT NULL DEFAULT 0,
-    create_by varchar(64) NOT NULL DEFAULT '',
-    update_at bigint NOT NULL DEFAULT 0,
-    update_by varchar(64) NOT NULL DEFAULT ''
-);
-
-CREATE TABLE notify_channel (
-    id bigserial PRIMARY KEY,
-    name varchar(255) NOT NULL,
-    ident varchar(255) NOT NULL,
-    description text,
-    enable smallint NOT NULL DEFAULT 0,
-    param_config text,
-    request_type varchar(50) NOT NULL,
-    request_config text,
-    weight int NOT NULL DEFAULT 0,
-    create_at bigint NOT NULL DEFAULT 0,
-    create_by varchar(64) NOT NULL DEFAULT '',
-    update_at bigint NOT NULL DEFAULT 0,
-    update_by varchar(64) NOT NULL DEFAULT ''
-);
-
-CREATE TABLE message_template (
-    id bigserial PRIMARY KEY,
-    name varchar(64) NOT NULL,
-    ident varchar(64) NOT NULL,
-    content text,
-    user_group_ids varchar(64),
-    notify_channel_ident varchar(64) NOT NULL DEFAULT '',
-    private int NOT NULL DEFAULT 0,
-    weight int NOT NULL DEFAULT 0,
-    create_at bigint NOT NULL DEFAULT 0,
-    create_by varchar(64) NOT NULL DEFAULT '',
-    update_at bigint NOT NULL DEFAULT 0,
-    update_by varchar(64) NOT NULL DEFAULT ''
-);
-
-CREATE TABLE event_pipeline (
-    id bigserial PRIMARY KEY,
-    name varchar(128) NOT NULL,
-    team_ids text,
-    description varchar(255) NOT NULL DEFAULT '',
-    filter_enable smallint NOT NULL DEFAULT 0,
-    label_filters text,
-    attribute_filters text,
-    processors text,
-    create_at bigint NOT NULL DEFAULT 0,
-    create_by varchar(64) NOT NULL DEFAULT '',
-    update_at bigint NOT NULL DEFAULT 0,
-    update_by varchar(64) NOT NULL DEFAULT ''
-);
-
-CREATE TABLE embedded_product (
-    id bigserial PRIMARY KEY,
-    name varchar(255) DEFAULT NULL,
-    url varchar(255) DEFAULT NULL,
-    is_private boolean DEFAULT NULL,
-    team_ids varchar(255),
-    create_at bigint NOT NULL DEFAULT 0,
-    create_by varchar(64) NOT NULL DEFAULT '',
-    update_at bigint NOT NULL DEFAULT 0,
-    update_by varchar(64) NOT NULL DEFAULT ''
 );

docker/initsql/a-n9e.sql

@@ -107,6 +107,12 @@ insert into `role_operation`(role_name, operation) values('Standard', '/help/mig
 insert into `role_operation`(role_name, operation) values('Standard', '/alert-rules-built-in');
 insert into `role_operation`(role_name, operation) values('Standard', '/dashboards-built-in');
 insert into `role_operation`(role_name, operation) values('Standard', '/trace/dependencies');
+
+insert into `role_operation`(role_name, operation) values('Admin', '/help/source');
+insert into `role_operation`(role_name, operation) values('Admin', '/help/sso');
+insert into `role_operation`(role_name, operation) values('Admin', '/help/notification-tpls');
+insert into `role_operation`(role_name, operation) values('Admin', '/help/notification-settings');
+
 insert into `role_operation`(role_name, operation) values('Standard', '/users');
 insert into `role_operation`(role_name, operation) values('Standard', '/user-groups');
 insert into `role_operation`(role_name, operation) values('Standard', '/user-groups/add');
@@ -465,7 +471,6 @@ CREATE TABLE `alert_cur_event` (
     `rule_config` text not null comment 'annotations',
     `tags` varchar(1024) not null default '' comment 'merge data_tags rule_tags, split by ,,',
     `original_tags` text comment 'labels key=val,,k2=v2',
-    `notify_rule_ids` text COMMENT 'notify rule ids',
     PRIMARY KEY (`id`),
     KEY (`hash`),
     KEY (`rule_id`),
@@ -508,7 +513,6 @@ CREATE TABLE `alert_his_event` (
     `original_tags` text comment 'labels key=val,,k2=v2',
     `annotations` text not null comment 'annotations',
     `rule_config` text not null comment 'annotations',
-    `notify_rule_ids` text COMMENT 'notify rule ids',
     PRIMARY KEY (`id`),
     INDEX `idx_last_eval_time` (`last_eval_time`),
     KEY (`hash`),
@@ -846,19 +850,6 @@ CREATE TABLE `event_pipeline` (
     PRIMARY KEY (`id`)
 ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4;
 
-CREATE TABLE `embedded_product` (
-    `id` bigint unsigned NOT NULL AUTO_INCREMENT,
-    `name` varchar(255) DEFAULT NULL,
-    `url` varchar(255) DEFAULT NULL,
-    `is_private` boolean DEFAULT NULL,
-    `team_ids` varchar(255),
-    `create_at` bigint not null default 0,
-    `create_by` varchar(64) not null default '',
-    `update_at` bigint not null default 0,
-    `update_by` varchar(64) not null default '',
-    PRIMARY KEY (`id`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-
 CREATE TABLE `task_meta`
 (
     `id`          bigint unsigned NOT NULL AUTO_INCREMENT,
@@ -2216,15 +2207,3 @@ CREATE TABLE task_host_99
     PRIMARY KEY (`ii`)
 ) ENGINE = InnoDB
   DEFAULT CHARSET = utf8mb4;
-
-CREATE TABLE `source_token` (
-    `id` bigint unsigned NOT NULL AUTO_INCREMENT,
-    `source_type` varchar(64) NOT NULL DEFAULT '' COMMENT 'source type',
-    `source_id` varchar(255) NOT NULL DEFAULT '' COMMENT 'source identifier',
-    `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'access token',
-    `expire_at` bigint NOT NULL DEFAULT 0 COMMENT 'expire timestamp',
-    `create_at` bigint NOT NULL DEFAULT 0 COMMENT 'create timestamp',
-    `create_by` varchar(64) NOT NULL DEFAULT '' COMMENT 'creator',
-    PRIMARY KEY (`id`),
-    KEY `idx_source_type_id_token` (`source_type`, `source_id`, `token`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

docker/migratesql/migrate.sql

@@ -244,21 +244,4 @@ CREATE TABLE `event_pipeline` (
     `update_at` bigint not null default 0,
     `update_by` varchar(64) not null default '',
     PRIMARY KEY (`id`)
-) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4;
-
-/* v8.0.0-next */
-CREATE TABLE `source_token` (
-    `id` bigint unsigned NOT NULL AUTO_INCREMENT,
-    `source_type` varchar(64) NOT NULL DEFAULT '' COMMENT 'source type',
-    `source_id` varchar(255) NOT NULL DEFAULT '' COMMENT 'source identifier',
-    `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'access token',
-    `expire_at` bigint NOT NULL DEFAULT 0 COMMENT 'expire timestamp',
-    `create_at` bigint NOT NULL DEFAULT 0 COMMENT 'create timestamp',
-    `create_by` varchar(64) NOT NULL DEFAULT '' COMMENT 'creator',
-    PRIMARY KEY (`id`),
-    KEY `idx_source_type_id_token` (`source_type`, `source_id`, `token`)
-) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-
-/* v8.0.0-beta.12 2025-06-03 */
-ALTER TABLE `alert_his_event` ADD COLUMN `notify_rule_ids` text COMMENT 'notify rule ids';
-ALTER TABLE `alert_cur_event` ADD COLUMN `notify_rule_ids` text COMMENT 'notify rule ids';
+) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4;

dscache/sync.go

@@ -9,8 +9,6 @@ import (
 	"github.com/ccfos/nightingale/v6/datasource"
 	_ "github.com/ccfos/nightingale/v6/datasource/ck"
 	"github.com/ccfos/nightingale/v6/datasource/es"
-	_ "github.com/ccfos/nightingale/v6/datasource/mysql"
-	_ "github.com/ccfos/nightingale/v6/datasource/postgresql"
 	"github.com/ccfos/nightingale/v6/dskit/tdengine"
 	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/ctx"

dskit/mysql/mysql.go

@@ -1,172 +0,0 @@
-// @Author: Ciusyan 5/10/24
-
-package mysql
-
-import (
-	"context"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"strings"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/dskit/pool"
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-
-	_ "github.com/go-sql-driver/mysql" // MySQL driver
-	"github.com/mitchellh/mapstructure"
-	"gorm.io/driver/mysql"
-	"gorm.io/gorm"
-)
-
-type MySQL struct {
-	Shards []Shard `json:"mysql.shards" mapstructure:"mysql.shards"`
-}
-
-type Shard struct {
-	Addr            string `json:"mysql.addr" mapstructure:"mysql.addr"`
-	DB              string `json:"mysql.db" mapstructure:"mysql.db"`
-	User            string `json:"mysql.user" mapstructure:"mysql.user"`
-	Password        string `json:"mysql.password" mapstructure:"mysql.password"`
-	Timeout         int    `json:"mysql.timeout" mapstructure:"mysql.timeout"`
-	MaxIdleConns    int    `json:"mysql.max_idle_conns" mapstructure:"mysql.max_idle_conns"`
-	MaxOpenConns    int    `json:"mysql.max_open_conns" mapstructure:"mysql.max_open_conns"`
-	ConnMaxLifetime int    `json:"mysql.conn_max_lifetime" mapstructure:"mysql.conn_max_lifetime"`
-	MaxQueryRows    int    `json:"mysql.max_query_rows" mapstructure:"mysql.max_query_rows"`
-}
-
-func NewMySQLWithSettings(ctx context.Context, settings interface{}) (*MySQL, error) {
-	newest := new(MySQL)
-	settingsMap := map[string]interface{}{}
-
-	switch s := settings.(type) {
-	case string:
-		if err := json.Unmarshal([]byte(s), &settingsMap); err != nil {
-			return nil, err
-		}
-	case map[string]interface{}:
-		settingsMap = s
-	default:
-		return nil, errors.New("unsupported settings type")
-	}
-
-	if err := mapstructure.Decode(settingsMap, newest); err != nil {
-		return nil, err
-	}
-
-	return newest, nil
-}
-
-// NewConn establishes a new connection to MySQL
-func (m *MySQL) NewConn(ctx context.Context, database string) (*gorm.DB, error) {
-	if len(m.Shards) == 0 {
-		return nil, errors.New("empty pgsql shards")
-	}
-
-	shard := m.Shards[0]
-
-	if shard.Timeout == 0 {
-		shard.Timeout = 300
-	}
-
-	if shard.MaxIdleConns == 0 {
-		shard.MaxIdleConns = 10
-	}
-
-	if shard.MaxOpenConns == 0 {
-		shard.MaxOpenConns = 100
-	}
-
-	if shard.ConnMaxLifetime == 0 {
-		shard.ConnMaxLifetime = 300
-	}
-
-	if shard.MaxQueryRows == 0 {
-		shard.MaxQueryRows = 100
-	}
-
-	if len(shard.Addr) == 0 {
-		return nil, errors.New("empty addr")
-	}
-
-	if len(shard.Addr) == 0 {
-		return nil, errors.New("empty addr")
-	}
-	var keys []string
-	var err error
-	keys = append(keys, shard.Addr)
-
-	keys = append(keys, shard.Password, shard.User)
-	if len(database) > 0 {
-		keys = append(keys, database)
-	}
-	cachedKey := strings.Join(keys, ":")
-	// cache conn with database
-	conn, ok := pool.PoolClient.Load(cachedKey)
-	if ok {
-		return conn.(*gorm.DB), nil
-	}
-	var db *gorm.DB
-	defer func() {
-		if db != nil && err == nil {
-			pool.PoolClient.Store(cachedKey, db)
-		}
-	}()
-
-	dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8&parseTime=True", shard.User, shard.Password, shard.Addr, database)
-
-	return sqlbase.NewDB(
-		ctx,
-		mysql.Open(dsn),
-		shard.MaxIdleConns,
-		shard.MaxOpenConns,
-		time.Duration(shard.ConnMaxLifetime)*time.Second,
-	)
-}
-
-func (m *MySQL) ShowDatabases(ctx context.Context) ([]string, error) {
-	db, err := m.NewConn(ctx, "")
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.ShowDatabases(ctx, db, "SHOW DATABASES")
-}
-
-func (m *MySQL) ShowTables(ctx context.Context, database string) ([]string, error) {
-	db, err := m.NewConn(ctx, database)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.ShowTables(ctx, db, "SHOW TABLES")
-}
-
-func (m *MySQL) DescTable(ctx context.Context, database, table string) ([]*types.ColumnProperty, error) {
-	db, err := m.NewConn(ctx, database)
-	if err != nil {
-		return nil, err
-	}
-
-	query := fmt.Sprintf("DESCRIBE %s", table)
-	return sqlbase.DescTable(ctx, db, query)
-}
-
-func (m *MySQL) SelectRows(ctx context.Context, database, table, query string) ([]map[string]interface{}, error) {
-	db, err := m.NewConn(ctx, database)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.SelectRows(ctx, db, table, query)
-}
-
-func (m *MySQL) ExecQuery(ctx context.Context, database string, sql string) ([]map[string]interface{}, error) {
-	db, err := m.NewConn(ctx, database)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.ExecQuery(ctx, db, sql)
-}

dskit/mysql/mysql_test.go

@@ -1,129 +0,0 @@
-// @Author: Ciusyan 5/11/24
-
-package mysql
-
-import (
-	"context"
-	"testing"
-
-	"github.com/stretchr/testify/require"
-)
-
-func TestNewMySQLWithSettings(t *testing.T) {
-	tests := []struct {
-		name     string
-		settings interface{}
-		wantErr  bool
-	}{
-		{
-			name:     "valid string settings",
-			settings: `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`,
-			wantErr:  false,
-		},
-		{
-			name:     "invalid settings type",
-			settings: 12345,
-			wantErr:  true,
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			got, err := NewMySQLWithSettings(context.Background(), tt.settings)
-			if (err != nil) != tt.wantErr {
-				t.Errorf("NewMySQLWithSettings() error = %v, wantErr %v", err, tt.wantErr)
-			}
-			t.Log(got)
-		})
-	}
-}
-
-func TestNewConn(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	tests := []struct {
-		name     string
-		database string
-		wantErr  bool
-	}{
-		{
-			name:     "valid connection",
-			database: "db1",
-			wantErr:  false,
-		},
-	}
-
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			_, err := mysql.NewConn(ctx, tt.database)
-			if (err != nil) != tt.wantErr {
-				t.Errorf("NewConn() error = %v, wantErr %v", err, tt.wantErr)
-				return
-			}
-		})
-	}
-}
-
-func TestShowDatabases(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	databases, err := mysql.ShowDatabases(ctx)
-	require.NoError(t, err)
-	t.Log(databases)
-}
-
-func TestShowTables(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	tables, err := mysql.ShowTables(ctx, "db1")
-	require.NoError(t, err)
-	t.Log(tables)
-}
-
-func TestDescTable(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	descTable, err := mysql.DescTable(ctx, "db1", "students")
-	require.NoError(t, err)
-	for _, desc := range descTable {
-		t.Logf("%+v", *desc)
-	}
-}
-
-func TestExecQuery(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	rows, err := mysql.ExecQuery(ctx, "db1", "SELECT * FROM students WHERE id = 10008")
-	require.NoError(t, err)
-	for _, row := range rows {
-		t.Log(row)
-	}
-}
-
-func TestSelectRows(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	rows, err := mysql.SelectRows(ctx, "db1", "students", "id > 10008")
-	require.NoError(t, err)
-	for _, row := range rows {
-		t.Log(row)
-	}
-}

dskit/mysql/timeseries.go

@@ -1,74 +0,0 @@
-package mysql
-
-import (
-	"context"
-	"fmt"
-	"strings"
-
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-
-	"gorm.io/gorm"
-)
-
-// Query executes a given SQL query in MySQL and returns the results
-func (m *MySQL) Query(ctx context.Context, query *sqlbase.QueryParam) ([]map[string]interface{}, error) {
-	db, err := m.NewConn(ctx, "")
-	if err != nil {
-		return nil, err
-	}
-
-	err = m.CheckMaxQueryRows(db, ctx, query)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.Query(ctx, db, query)
-}
-
-// QueryTimeseries executes a time series data query using the given parameters
-func (m *MySQL) QueryTimeseries(ctx context.Context, query *sqlbase.QueryParam) ([]types.MetricValues, error) {
-	db, err := m.NewConn(ctx, "")
-	if err != nil {
-		return nil, err
-	}
-
-	err = m.CheckMaxQueryRows(db, ctx, query)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.QueryTimeseries(ctx, db, query)
-}
-
-func (m *MySQL) CheckMaxQueryRows(db *gorm.DB, ctx context.Context, query *sqlbase.QueryParam) error {
-	sql := strings.ReplaceAll(query.Sql, ";", "")
-	checkQuery := &sqlbase.QueryParam{
-		Sql: fmt.Sprintf("SELECT COUNT(*) as count FROM (%s) AS subquery;", sql),
-	}
-
-	res, err := sqlbase.Query(ctx, db, checkQuery)
-	if err != nil {
-		return err
-	}
-
-	if len(res) > 0 {
-		if count, exists := res[0]["count"]; exists {
-			v, err := sqlbase.ParseFloat64Value(count)
-			if err != nil {
-				return err
-			}
-
-			maxQueryRows := m.Shards[0].MaxQueryRows
-			if maxQueryRows == 0 {
-				maxQueryRows = 500
-			}
-
-			if v > float64(maxQueryRows) {
-				return fmt.Errorf("query result rows count %d exceeds the maximum limit %d", int(v), maxQueryRows)
-			}
-		}
-	}
-
-	return nil
-}

dskit/mysql/timeseries_test.go

@@ -1,62 +0,0 @@
-// @Author: Ciusyan 5/11/24
-
-package mysql
-
-import (
-	"context"
-	"testing"
-
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-
-	"github.com/stretchr/testify/require"
-)
-
-func TestQuery(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	param := &sqlbase.QueryParam{
-		Sql: "SELECT * FROM students WHERE id > 10900",
-		Keys: types.Keys{
-			ValueKey:   "",
-			LabelKey:   "",
-			TimeKey:    "",
-			TimeFormat: "",
-		},
-	}
-
-	rows, err := mysql.Query(ctx, param)
-	require.NoError(t, err)
-	for _, row := range rows {
-		t.Log(row)
-	}
-}
-
-func TestQueryTimeseries(t *testing.T) {
-	ctx := context.Background()
-	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
-	mysql, err := NewMySQLWithSettings(ctx, settings)
-	require.NoError(t, err)
-
-	// Prepare a test query parameter
-	param := &sqlbase.QueryParam{
-		Sql: "SELECT id, grade, student_name, a_grade, update_time FROM students WHERE grade > 20000", // Modify SQL query to select specific columns
-		Keys: types.Keys{
-			ValueKey:   "grade a_grade",                 // Set the value key to the column name containing the metric value
-			LabelKey:   "id student_name",               // Set the label key to the column name containing the metric label
-			TimeKey:    "update_time",                   // Set the time key to the column name containing the timestamp
-			TimeFormat: "2006-01-02 15:04:05 +0000 UTC", // Provide the time format according to the timestamp column's format
-		},
-	}
-
-	// Execute the query and retrieve the time series data
-	metricValues, err := mysql.QueryTimeseries(ctx, param)
-	require.NoError(t, err)
-
-	for _, metric := range metricValues {
-		t.Log(metric)
-	}
-}

dskit/pool/pool.go

@@ -1,37 +0,0 @@
-package pool
-
-import (
-	"bytes"
-	"sync"
-	"time"
-
-	gc "github.com/patrickmn/go-cache"
-)
-
-var (
-	PoolClient = new(sync.Map)
-)
-
-var (
-	// default cache instance, do not use this if you want to specify the defaultExpiration
-	DefaultCache = gc.New(time.Hour*24, time.Hour)
-)
-
-var (
-	bytesPool = sync.Pool{
-		New: func() interface{} { return new(bytes.Buffer) },
-	}
-)
-
-func PoolGetBytesBuffer() *bytes.Buffer {
-	buf := bytesPool.Get().(*bytes.Buffer)
-	buf.Reset()
-	return buf
-}
-
-func PoolPutBytesBuffer(buf *bytes.Buffer) {
-	if buf == nil {
-		return
-	}
-	bytesPool.Put(buf)
-}

dskit/postgres/postgres.go

@@ -1,207 +0,0 @@
-// @Author: Ciusyan 5/20/24
-
-package postgres
-
-import (
-	"context"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"strings"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/dskit/pool"
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-
-	_ "github.com/lib/pq" // PostgreSQL driver
-	"github.com/mitchellh/mapstructure"
-	"gorm.io/driver/postgres"
-	"gorm.io/gorm"
-)
-
-type PostgreSQL struct {
-	Shard `json:",inline"  mapstructure:",squash"`
-}
-
-type Shard struct {
-	Addr            string `json:"pgsql.addr" mapstructure:"pgsql.addr"`
-	DB              string `json:"pgsql.db" mapstructure:"pgsql.db"`
-	User            string `json:"pgsql.user" mapstructure:"pgsql.user"`
-	Password        string `json:"pgsql.password" mapstructure:"pgsql.password" `
-	Timeout         int    `json:"pgsql.timeout" mapstructure:"pgsql.timeout"`
-	MaxIdleConns    int    `json:"pgsql.max_idle_conns" mapstructure:"pgsql.max_idle_conns"`
-	MaxOpenConns    int    `json:"pgsql.max_open_conns" mapstructure:"pgsql.max_open_conns"`
-	ConnMaxLifetime int    `json:"pgsql.conn_max_lifetime" mapstructure:"pgsql.conn_max_lifetime"`
-	MaxQueryRows    int    `json:"pgsql.max_query_rows" mapstructure:"pgsql.max_query_rows"`
-}
-
-// NewPostgreSQLWithSettings initializes a new PostgreSQL instance with the given settings
-func NewPostgreSQLWithSettings(ctx context.Context, settings interface{}) (*PostgreSQL, error) {
-	newest := new(PostgreSQL)
-	settingsMap := map[string]interface{}{}
-
-	switch s := settings.(type) {
-	case string:
-		if err := json.Unmarshal([]byte(s), &settingsMap); err != nil {
-			return nil, err
-		}
-	case map[string]interface{}:
-		settingsMap = s
-	case *PostgreSQL:
-		return s, nil
-	case PostgreSQL:
-		return &s, nil
-	case Shard:
-		newest.Shard = s
-		return newest, nil
-	case *Shard:
-		newest.Shard = *s
-		return newest, nil
-	default:
-		return nil, errors.New("unsupported settings type")
-	}
-
-	if err := mapstructure.Decode(settingsMap, newest); err != nil {
-		return nil, err
-	}
-
-	return newest, nil
-}
-
-// NewConn establishes a new connection to PostgreSQL
-func (p *PostgreSQL) NewConn(ctx context.Context, database string) (*gorm.DB, error) {
-	if len(p.DB) == 0 && len(database) == 0 {
-		return nil, errors.New("empty pgsql database") // 兼容阿里实时数仓Holgres, 连接时必须指定db名字
-	}
-
-	if p.Shard.Timeout == 0 {
-		p.Shard.Timeout = 60
-	}
-
-	if p.Shard.MaxIdleConns == 0 {
-		p.Shard.MaxIdleConns = 10
-	}
-
-	if p.Shard.MaxOpenConns == 0 {
-		p.Shard.MaxOpenConns = 100
-	}
-
-	if p.Shard.ConnMaxLifetime == 0 {
-		p.Shard.ConnMaxLifetime = 14400
-	}
-
-	if len(p.Shard.Addr) == 0 {
-		return nil, errors.New("empty fe-node addr")
-	}
-	var keys []string
-	var err error
-	keys = append(keys, p.Shard.Addr)
-
-	keys = append(keys, p.Shard.Password, p.Shard.User)
-	if len(database) > 0 {
-		keys = append(keys, database)
-	}
-	cachedKey := strings.Join(keys, ":")
-	// cache conn with database
-	conn, ok := pool.PoolClient.Load(cachedKey)
-	if ok {
-		return conn.(*gorm.DB), nil
-	}
-
-	var db *gorm.DB
-	defer func() {
-		if db != nil && err == nil {
-			pool.PoolClient.Store(cachedKey, db)
-		}
-	}()
-
-	// Simplified connection logic for PostgreSQL
-	dsn := fmt.Sprintf("postgres://%s:%s@%s/%s?sslmode=disable&TimeZone=Asia/Shanghai", p.Shard.User, p.Shard.Password, p.Shard.Addr, database)
-	db, err = sqlbase.NewDB(
-		ctx,
-		postgres.Open(dsn),
-		p.Shard.MaxIdleConns,
-		p.Shard.MaxOpenConns,
-		time.Duration(p.Shard.ConnMaxLifetime)*time.Second,
-	)
-
-	if err != nil {
-		if db != nil {
-			sqlDB, _ := db.DB()
-			if sqlDB != nil {
-				sqlDB.Close()
-			}
-		}
-		return nil, err
-	}
-
-	return db, nil
-}
-
-// ShowDatabases lists all databases in PostgreSQL
-func (p *PostgreSQL) ShowDatabases(ctx context.Context, searchKeyword string) ([]string, error) {
-	db, err := p.NewConn(ctx, "postgres")
-	if err != nil {
-		return nil, err
-	}
-	sql := fmt.Sprintf("SELECT datname FROM pg_database WHERE datistemplate = false AND datname LIKE %s",
-		"'%"+searchKeyword+"%'")
-	return sqlbase.ShowDatabases(ctx, db, sql)
-}
-
-// ShowTables lists all tables in a given database
-func (p *PostgreSQL) ShowTables(ctx context.Context, searchKeyword string) (map[string][]string, error) {
-	db, err := p.NewConn(ctx, p.DB)
-	if err != nil {
-		return nil, err
-	}
-	sql := fmt.Sprintf("SELECT schemaname, tablename FROM pg_tables WHERE schemaname !='information_schema' and schemaname !='pg_catalog'  and  tablename LIKE %s",
-		"'%"+searchKeyword+"%'")
-	rets, err := sqlbase.ExecQuery(ctx, db, sql)
-	if err != nil {
-		return nil, err
-	}
-	tabs := make(map[string][]string, 3)
-	for _, row := range rets {
-		if val, ok := row["schemaname"].(string); ok {
-			tabs[val] = append(tabs[val], row["tablename"].(string))
-		}
-	}
-	return tabs, nil
-}
-
-// DescTable describes the schema of a specified table in PostgreSQL
-// scheme default: public if not specified
-func (p *PostgreSQL) DescTable(ctx context.Context, scheme, table string) ([]*types.ColumnProperty, error) {
-	db, err := p.NewConn(ctx, p.DB)
-	if err != nil {
-		return nil, err
-	}
-	if scheme == "" {
-		scheme = "public"
-	}
-
-	query := fmt.Sprintf("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = '%s' AND table_schema = '%s'", table, scheme)
-	return sqlbase.DescTable(ctx, db, query)
-}
-
-// SelectRows selects rows from a specified table in PostgreSQL based on a given query
-func (p *PostgreSQL) SelectRows(ctx context.Context, table, where string) ([]map[string]interface{}, error) {
-	db, err := p.NewConn(ctx, p.DB)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.SelectRows(ctx, db, table, where)
-}
-
-// ExecQuery executes a SQL query in PostgreSQL
-func (p *PostgreSQL) ExecQuery(ctx context.Context, sql string) ([]map[string]interface{}, error) {
-	db, err := p.NewConn(ctx, p.DB)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.ExecQuery(ctx, db, sql)
-}

dskit/postgres/timeseries.go

@@ -1,73 +0,0 @@
-package postgres
-
-import (
-	"context"
-	"fmt"
-	"strings"
-
-	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
-	"github.com/ccfos/nightingale/v6/dskit/types"
-	"gorm.io/gorm"
-)
-
-// Query executes a given SQL query in PostgreSQL and returns the results
-func (p *PostgreSQL) Query(ctx context.Context, query *sqlbase.QueryParam) ([]map[string]interface{}, error) {
-	db, err := p.NewConn(ctx, p.Shard.DB)
-	if err != nil {
-		return nil, err
-	}
-
-	err = p.CheckMaxQueryRows(db, ctx, query)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.Query(ctx, db, query)
-}
-
-// QueryTimeseries executes a time series data query using the given parameters
-func (p *PostgreSQL) QueryTimeseries(ctx context.Context, query *sqlbase.QueryParam) ([]types.MetricValues, error) {
-	db, err := p.NewConn(ctx, p.Shard.DB)
-	if err != nil {
-		return nil, err
-	}
-
-	err = p.CheckMaxQueryRows(db, ctx, query)
-	if err != nil {
-		return nil, err
-	}
-
-	return sqlbase.QueryTimeseries(ctx, db, query, true)
-}
-
-func (p *PostgreSQL) CheckMaxQueryRows(db *gorm.DB, ctx context.Context, query *sqlbase.QueryParam) error {
-	sql := strings.ReplaceAll(query.Sql, ";", "")
-	checkQuery := &sqlbase.QueryParam{
-		Sql: fmt.Sprintf("SELECT COUNT(*) as count FROM (%s) AS subquery;", sql),
-	}
-
-	res, err := sqlbase.Query(ctx, db, checkQuery)
-	if err != nil {
-		return err
-	}
-
-	if len(res) > 0 {
-		if count, exists := res[0]["count"]; exists {
-			v, err := sqlbase.ParseFloat64Value(count)
-			if err != nil {
-				return err
-			}
-
-			maxQueryRows := p.Shard.MaxQueryRows
-			if maxQueryRows == 0 {
-				maxQueryRows = 500
-			}
-
-			if v > float64(maxQueryRows) {
-				return fmt.Errorf("query result rows count %d exceeds the maximum limit %d", int(v), maxQueryRows)
-			}
-		}
-	}
-
-	return nil
-}

dskit/sqlbase/base.go

@@ -9,9 +9,9 @@ import (
 	"strings"
 	"time"
 
-	"gorm.io/gorm"
-
 	"github.com/ccfos/nightingale/v6/dskit/types"
+
+	"gorm.io/gorm"
 )
 
 // NewDB creates a new Gorm DB instance based on the provided gorm.Dialector and configures the connection pool
@@ -19,7 +19,7 @@ func NewDB(ctx context.Context, dialector gorm.Dialector, maxIdleConns, maxOpenC
 	// Create a new Gorm DB instance
 	db, err := gorm.Open(dialector, &gorm.Config{})
 	if err != nil {
-		return db, err
+		return nil, err
 	}
 
 	// Configure the connection pool
@@ -35,17 +35,6 @@ func NewDB(ctx context.Context, dialector gorm.Dialector, maxIdleConns, maxOpenC
 	return db.WithContext(ctx), sqlDB.Ping()
 }
 
-func CloseDB(db *gorm.DB) error {
-	if db != nil {
-		sqlDb, err := db.DB()
-		if err != nil {
-			return err
-		}
-		return sqlDb.Close()
-	}
-	return nil
-}
-
 // ShowTables retrieves a list of all tables in the specified database
 func ShowTables(ctx context.Context, db *gorm.DB, query string) ([]string, error) {
 	var tables []string
@@ -123,7 +112,7 @@ func DescTable(ctx context.Context, db *gorm.DB, query string) ([]*types.ColumnP
 		}
 
 		// Convert the database-specific type to internal type
-		type2, indexable := ConvertDBType(db.Dialector.Name(), typ)
+		type2, indexable := convertDBType(db.Dialector.Name(), typ)
 		columns = append(columns, &types.ColumnProperty{
 			Field:     field,
 			Type:      typ,
@@ -186,7 +175,7 @@ func SelectRows(ctx context.Context, db *gorm.DB, table, query string) ([]map[st
 }
 
 // convertDBType converts MySQL or PostgreSQL data types to custom internal types and determines if they are indexable
-func ConvertDBType(dialect, dbType string) (string, bool) {
+func convertDBType(dialect, dbType string) (string, bool) {
 	typ := strings.ToLower(dbType)
 
 	// Common type conversions
@@ -201,7 +190,7 @@ func ConvertDBType(dialect, dbType string) (string, bool) {
 		strings.HasPrefix(typ, "char"), strings.HasPrefix(typ, "tinytext"),
 		strings.HasPrefix(typ, "mediumtext"), strings.HasPrefix(typ, "longtext"),
 		strings.HasPrefix(typ, "character varying"), strings.HasPrefix(typ, "nvarchar"),
-		strings.HasPrefix(typ, "nchar"), strings.HasPrefix(typ, "bpchar"):
+		strings.HasPrefix(typ, "nchar"):
 		return types.LogExtractValueTypeText, true
 
 	case strings.HasPrefix(typ, "float"), strings.HasPrefix(typ, "double"),
@@ -214,7 +203,7 @@ func ConvertDBType(dialect, dbType string) (string, bool) {
 		strings.HasPrefix(typ, "time"), strings.HasPrefix(typ, "smalldatetime"):
 		return types.LogExtractValueTypeDate, false
 
-	case strings.HasPrefix(typ, "boolean"), strings.HasPrefix(typ, "bit"), strings.HasPrefix(typ, "bool"):
+	case strings.HasPrefix(typ, "boolean"), strings.HasPrefix(typ, "bit"):
 		return types.LogExtractValueTypeBool, false
 	}
 

dskit/sqlbase/timeseries.go

@@ -112,8 +112,7 @@ func FormatMetricValues(keys types.Keys, rows []map[string]interface{}, ignoreDe
 				metricTs[k] = float64(ts.Unix())
 			default:
 				// Default to labels for any unrecognized columns
-				if !ignore && keys.LabelKey == "" {
-					// 只有当 labelKey 为空时，才将剩余的列作为 label
+				if !ignore {
 					labels[k] = fmt.Sprintf("%v", v)
 				}
 			}

go.mod

@@ -27,13 +27,11 @@ require (
 	github.com/jinzhu/copier v0.4.0
 	github.com/json-iterator/go v1.1.12
 	github.com/koding/multiconfig v0.0.0-20171124222453-69c27309b2d7
-	github.com/lib/pq v1.0.0
 	github.com/mailru/easyjson v0.7.7
 	github.com/mattn/go-isatty v0.0.19
 	github.com/mitchellh/mapstructure v1.5.0
 	github.com/mojocn/base64Captcha v1.3.6
 	github.com/olivere/elastic/v7 v7.0.32
-	github.com/patrickmn/go-cache v2.1.0+incompatible
 	github.com/pelletier/go-toml/v2 v2.0.8
 	github.com/pkg/errors v0.9.1
 	github.com/prometheus/client_golang v1.20.5
@@ -122,7 +120,7 @@ require (
 	github.com/go-playground/locales v0.14.1 // indirect
 	github.com/go-playground/universal-translator v0.18.1 // indirect
 	github.com/go-playground/validator/v10 v10.14.0 // indirect
-	github.com/go-sql-driver/mysql v1.6.0
+	github.com/go-sql-driver/mysql v1.6.0 // indirect
 	github.com/goccy/go-json v0.10.2 // indirect
 	github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
 	github.com/grafana/regexp v0.0.0-20221122212121-6b5c0a4cb7fd // indirect

go.sum

@@ -220,8 +220,6 @@ github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+
 github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
 github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
 github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
-github.com/lib/pq v1.0.0 h1:X5PMW56eZitiTeO7tKzZxFCSpbFZJtkMMooicw2us9A=
-github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
 github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
@@ -248,8 +246,6 @@ github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
 github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
 github.com/olivere/elastic/v7 v7.0.32 h1:R7CXvbu8Eq+WlsLgxmKVKPox0oOwAE/2T9Si5BnvK6E=
 github.com/olivere/elastic/v7 v7.0.32/go.mod h1:c7PVmLe3Fxq77PIfY/bZmxY/TAamBhCzZ8xDOE09a9k=
-github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
-github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
 github.com/paulmach/orb v0.11.1 h1:3koVegMC4X/WeiXYz9iswopaTwMem53NzTJuTF20JzU=
 github.com/paulmach/orb v0.11.1/go.mod h1:5mULz1xQfs3bmQm63QEJA6lNGujuRafwA5S/EnuLaLU=
 github.com/paulmach/protoscan v0.2.1/go.mod h1:SpcSwydNLrxUGSDvXvO0P7g7AuhJ7lcKfDlhJCDw2gY=

integrations/Linux/dashboards/categraf-processes.json

@@ -1,12 +1,18 @@
 {
-    "name": "机器进程数量统计（使用 Categraf 作为采集器）",
-    "tags": "Categraf",
+    "id": 0,
+    "group_id": 0,
+    "name": "Processes by UlricQin",
     "ident": "",
-    "uuid": 1717556327738575000,
+    "tags": "Categraf Linux OS",
+    "create_at": 0,
+    "create_by": "",
+    "update_at": 0,
+    "update_by": "",
     "configs": {
         "panels": [
             {
                 "custom": {
+                    "baseColor": "#9470FF",
                     "calc": "lastNotNull",
                     "serieWidth": 20,
                     "sortOrder": "desc"
@@ -35,17 +41,7 @@
                             },
                             "type": "range"
                         }
-                    ],
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "#9470FF",
-                                "type": "base",
-                                "value": null
-                            }
-                        ]
-                    }
+                    ]
                 },
                 "targets": [
                     {
@@ -66,6 +62,7 @@
             },
             {
                 "custom": {
+                    "baseColor": "#9470FF",
                     "calc": "lastNotNull",
                     "serieWidth": 20,
                     "sortOrder": "desc"
@@ -94,17 +91,7 @@
                             },
                             "type": "range"
                         }
-                    ],
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "#9470FF",
-                                "type": "base",
-                                "value": null
-                            }
-                        ]
-                    }
+                    ]
                 },
                 "targets": [
                     {
@@ -125,6 +112,7 @@
             },
             {
                 "custom": {
+                    "baseColor": "#9470FF",
                     "calc": "lastNotNull",
                     "serieWidth": 20,
                     "sortOrder": "desc"
@@ -162,17 +150,7 @@
                             },
                             "type": "range"
                         }
-                    ],
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "#9470FF",
-                                "type": "base",
-                                "value": null
-                            }
-                        ]
-                    }
+                    ]
                 },
                 "targets": [
                     {
@@ -238,26 +216,30 @@
         ],
         "var": [
             {
+                "definition": "prometheus",
+                "label": "",
                 "name": "Datasource",
-                "label": "数据源",
-                "type": "datasource",
-                "hide": false,
-                "definition": "prometheus"
+                "type": "datasource"
             },
             {
-                "name": "ident",
-                "label": "机器",
-                "type": "query",
-                "hide": false,
+                "allOption": true,
                 "datasource": {
                     "cate": "prometheus",
                     "value": "${Datasource}"
                 },
                 "definition": "label_values(processes_running, ident)",
+                "label": "Host",
                 "multi": true,
-                "allOption": true
+                "name": "ident",
+                "type": "query"
             }
         ],
         "version": "3.0.0"
-    }
+    },
+    "public": 0,
+    "public_cate": 0,
+    "bgids": null,
+    "built_in": 0,
+    "hide": 0,
+    "uuid": 1717556327738575000
 }

integrations/Linux/dashboards/host_generic_categraf_all.json

@@ -1,6 +1,6 @@
 {
-    "name": "机器常用指标（使用 Categraf 作为采集器，如果只想看当前业务组内的机器修改大盘变量 ident 的变量类型为机器标识即可）",
-    "tags": "Categraf",
+    "name": "机器常用指标 - 所有机器",
+    "tags": "categraf",
     "ident": "",
     "uuid": 1737103014612000,
     "configs": {
@@ -33,11 +33,9 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "count(last_over_time(system_uptime{ident=~\"$ident\"}[$__rate_interval]))",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15,
-                        "instant": false
+                        "expr": "count(last_over_time(system_uptime{ident=~\"$ident\"}[1m]))",
+                        "maxDataPoints": 240,
+                        "refId": "A"
                     }
                 ],
                 "transformations": [
@@ -50,7 +48,7 @@
                 "maxPerRow": 4,
                 "custom": {
                     "textMode": "value",
-                    "graphMode": "area",
+                    "graphMode": "none",
                     "colorMode": "background",
                     "calc": "lastNotNull",
                     "valueField": "Value",
@@ -335,7 +333,7 @@
                     {
                         "expr": "100-cpu_usage_idle{ident=~\"$ident\",cpu=\"cpu-total\"}",
                         "legend": "{{ident}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -350,8 +348,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -431,9 +428,9 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "rate(diskio_io_time{ident=~\"$ident\"}[$__rate_interval])/10",
+                        "expr": "rate(diskio_io_time{ident=~\"$ident\"}[1m])/10",
                         "legend": "{{ident}} {{name}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -448,8 +445,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -531,7 +527,7 @@
                     {
                         "expr": "mem_used_percent{ident=~\"$ident\"}",
                         "legend": "{{ident}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -546,8 +542,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -627,9 +622,9 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "(1 - mem_swap_free{ident=~\"$ident\"} / mem_swap_total{ident=~\"$ident\"})*100 and mem_swap_total{ident=~\"$ident\"} > 0",
+                        "expr": "(1 - mem_swap_free / mem_swap_total)*100",
                         "legend": "{{ident}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -645,8 +640,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -726,9 +720,9 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "rate(kernel_vmstat_oom_kill{ident=~\"$ident\"}[$__rate_interval])",
+                        "expr": "increase(kernel_vmstat_oom_kill{ident=~\"$ident\"}[5m])",
                         "legend": "{{ident}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -739,12 +733,11 @@
                         "options": {}
                     }
                 ],
-                "name": "每秒OOM次数",
+                "name": "5分钟内OOM次数",
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -769,8 +762,8 @@
                         "steps": [
                             {
                                 "color": "#634CD9",
-                                "value": null,
-                                "type": "base"
+                                "type": "base",
+                                "value": null
                             }
                         ]
                     },
@@ -834,9 +827,9 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "rate(net_bytes_recv{ident=~\"$ident\"}[$__rate_interval])*8",
+                        "expr": "rate(net_bytes_recv{ident=~\"$ident\"}[1m])*8",
                         "legend": "{{ident}} {{interface}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -851,8 +844,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -922,9 +914,9 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "rate(net_bytes_sent{ident=~\"$ident\"}[$__rate_interval])*8",
+                        "expr": "rate(net_bytes_sent{ident=~\"$ident\"}[1m])*8",
                         "legend": "{{ident}} {{interface}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "B",
                         "step": 15
                     }
@@ -939,8 +931,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -995,7 +986,21 @@
                 ]
             },
             {
-                "type": "timeseries",
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.03,
+                    "gradientMode": "none",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "scaleDistribution": {
+                        "type": "linear"
+                    },
+                    "spanNulls": false,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "description": "",
                 "id": "cfb80689-de7b-47fb-9155-052b796dd7f5",
                 "layout": {
                     "h": 5,
@@ -1005,68 +1010,29 @@
                     "i": "cfb80689-de7b-47fb-9155-052b796dd7f5",
                     "isResizable": true
                 },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "netstat_tcp_tw{ident=~\"$ident\"}",
-                        "maxDataPoints": 480,
-                        "refId": "B",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "Time Wait 状态的连接数",
-                "description": "",
                 "maxPerRow": 4,
+                "name": "Time Wait 状态的连接数",
                 "options": {
-                    "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
-                    },
                     "legend": {
-                        "displayMode": "hidden",
-                        "placement": "bottom",
                         "behaviour": "showItem",
-                        "selectMode": "single"
+                        "displayMode": "hidden"
                     },
                     "standardOptions": {
                         "decimals": 0
                     },
                     "thresholds": {
-                        "mode": "absolute",
                         "steps": [
                             {
                                 "color": "#634CD9",
-                                "value": null,
-                                "type": "base"
+                                "type": "base",
+                                "value": null
                             }
                         ]
                     },
-                    "thresholdsStyle": {
-                        "mode": "dashed"
+                    "tooltip": {
+                        "mode": "single"
                     }
                 },
-                "custom": {
-                    "drawStyle": "lines",
-                    "lineInterpolation": "smooth",
-                    "spanNulls": false,
-                    "lineWidth": 2,
-                    "fillOpacity": 0.03,
-                    "gradientMode": "none",
-                    "stack": "off",
-                    "scaleDistribution": {
-                        "type": "linear"
-                    },
-                    "showPoints": "none",
-                    "pointSize": 5
-                },
                 "overrides": [
                     {
                         "matcher": {
@@ -1076,7 +1042,23 @@
                             "rightYAxisDisplay": "off"
                         }
                     }
-                ]
+                ],
+                "targets": [
+                    {
+                        "expr": "netstat_tcp_tw{ident=~\"$ident\"}",
+                        "maxDataPoints": 240,
+                        "refId": "B",
+                        "step": 15
+                    }
+                ],
+                "transformations": [
+                    {
+                        "id": "organize",
+                        "options": {}
+                    }
+                ],
+                "type": "timeseries",
+                "version": "3.0.0"
             },
             {
                 "type": "timeseries",
@@ -1094,18 +1076,16 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "rate(net_err_in{ident=~\"$ident\"}[$__rate_interval])",
+                        "expr": "rate(net_err_in{ident=~\"$ident\"}[1m])",
                         "legend": "{{ident}}-{{interface}}-in",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
+                        "maxDataPoints": 240,
+                        "refId": "A"
                     },
                     {
-                        "expr": "rate(net_err_out{ident=~\"$ident\"}[$__rate_interval])",
+                        "expr": "rate(net_err_out{ident=~\"$ident\"}[1m])",
                         "legend": "{{ident}}-{{interface}}-out",
-                        "maxDataPoints": 480,
-                        "refId": "B",
-                        "step": 15
+                        "maxDataPoints": 240,
+                        "refId": "B"
                     }
                 ],
                 "transformations": [
@@ -1118,8 +1098,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -1185,18 +1164,16 @@
                 "datasourceValue": "${prom}",
                 "targets": [
                     {
-                        "expr": "rate(net_drop_in{ident=~\"$ident\"}[$__rate_interval])",
+                        "expr": "rate(net_drop_in{ident=~\"$ident\"}[1m])",
                         "legend": "{{ident}}-{{interface}}-in",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
+                        "maxDataPoints": 240,
+                        "refId": "A"
                     },
                     {
-                        "expr": "rate(net_drop_out{ident=~\"$ident\"}[$__rate_interval])",
+                        "expr": "rate(net_drop_out{ident=~\"$ident\"}[1m])",
                         "legend": "{{ident}}-{{interface}}-out",
-                        "maxDataPoints": 480,
-                        "refId": "B",
-                        "step": 15
+                        "maxDataPoints": 240,
+                        "refId": "B"
                     }
                 ],
                 "transformations": [
@@ -1209,8 +1186,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -1293,7 +1269,7 @@
                     {
                         "expr": "disk_device_error{ident=~\"$ident\"}",
                         "legend": "{{ident}} {{path}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     }
@@ -1308,8 +1284,7 @@
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -1318,7 +1293,7 @@
                         "selectMode": "single"
                     },
                     "standardOptions": {
-                        "decimals": 0
+                        "decimals": 2
                     },
                     "thresholds": {
                         "mode": "absolute",
@@ -1377,7 +1352,7 @@
                     {
                         "expr": "100 * conntrack_ip_conntrack_count{ident=~\"$ident\"} / conntrack_ip_conntrack_max{ident=~\"$ident\"}",
                         "legend": "ip_conntrack {{ident}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "A",
                         "step": 15
                     },
@@ -1385,7 +1360,7 @@
                         "__mode__": "__query__",
                         "expr": "100 * conntrack_nf_conntrack_count{ident=~\"$ident\"} / conntrack_nf_conntrack_max{ident=~\"$ident\"}",
                         "legend": "nf_conntrack {{ident}}",
-                        "maxDataPoints": 480,
+                        "maxDataPoints": 240,
                         "refId": "B",
                         "step": 15
                     }
@@ -1397,12 +1372,10 @@
                     }
                 ],
                 "name": "Conntrack使用率",
-                "description": "`dmesg -T` 有时看到 conntrack table full 的报错，大概率就是 conntrack 限制太小了，需要调整内核参数",
                 "maxPerRow": 4,
                 "options": {
                     "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
+                        "mode": "single"
                     },
                     "legend": {
                         "displayMode": "hidden",
@@ -1452,346 +1425,6 @@
                         }
                     }
                 ]
-            },
-            {
-                "type": "timeseries",
-                "id": "7c90380f-5ab6-4aa5-9070-f604985a0389",
-                "layout": {
-                    "h": 5,
-                    "w": 12,
-                    "x": 0,
-                    "y": 32,
-                    "i": "e7117d7c-b946-49fa-bc49-2afb0d2b3a44",
-                    "isResizable": true
-                },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "processes_total{ident=~\"$ident\"}",
-                        "legend": "",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "Process 总量",
-                "maxPerRow": 4,
-                "options": {
-                    "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
-                    },
-                    "legend": {
-                        "displayMode": "hidden",
-                        "placement": "bottom",
-                        "behaviour": "showItem",
-                        "selectMode": "single"
-                    },
-                    "standardOptions": {
-                        "decimals": 0
-                    },
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "rgba(44, 157, 61, 1)",
-                                "value": null,
-                                "type": "base"
-                            }
-                        ]
-                    },
-                    "thresholdsStyle": {
-                        "mode": "dashed"
-                    }
-                },
-                "custom": {
-                    "drawStyle": "lines",
-                    "lineInterpolation": "smooth",
-                    "spanNulls": false,
-                    "lineWidth": 2,
-                    "fillOpacity": 0.03,
-                    "gradientMode": "none",
-                    "stack": "off",
-                    "scaleDistribution": {
-                        "type": "linear"
-                    },
-                    "showPoints": "none",
-                    "pointSize": 5
-                },
-                "overrides": [
-                    {
-                        "matcher": {
-                            "id": "byFrameRefID"
-                        },
-                        "properties": {
-                            "rightYAxisDisplay": "off"
-                        }
-                    }
-                ]
-            },
-            {
-                "type": "timeseries",
-                "id": "3334c222-dd92-49eb-9744-4ce0f59031e4",
-                "layout": {
-                    "h": 5,
-                    "w": 12,
-                    "x": 12,
-                    "y": 32,
-                    "i": "0ecb9f26-4c4d-40d7-9934-5116e3ffa51a",
-                    "isResizable": true
-                },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "procstat_rlimit_num_fds_hard{ident=~\"$ident\"}",
-                        "legend": "",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "进程句柄数限制（低于4096要注意）",
-                "description": "以现在的硬件配置，通常句柄的 ulimit 应该比较大，如果低于 4096，大概率是忘记修改配置了，需要注意。这个数据是 Categraf 的 procstat 插件采集的。",
-                "maxPerRow": 4,
-                "options": {
-                    "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
-                    },
-                    "legend": {
-                        "displayMode": "hidden",
-                        "placement": "bottom",
-                        "behaviour": "showItem",
-                        "selectMode": "single"
-                    },
-                    "standardOptions": {
-                        "decimals": 0
-                    },
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "rgba(44, 157, 61, 1)",
-                                "value": null,
-                                "type": "base"
-                            }
-                        ]
-                    },
-                    "thresholdsStyle": {
-                        "mode": "dashed"
-                    }
-                },
-                "custom": {
-                    "drawStyle": "lines",
-                    "lineInterpolation": "smooth",
-                    "spanNulls": false,
-                    "lineWidth": 2,
-                    "fillOpacity": 0.03,
-                    "gradientMode": "none",
-                    "stack": "off",
-                    "scaleDistribution": {
-                        "type": "linear"
-                    },
-                    "showPoints": "none",
-                    "pointSize": 5
-                },
-                "overrides": [
-                    {
-                        "matcher": {
-                            "id": "byFrameRefID"
-                        },
-                        "properties": {
-                            "rightYAxisDisplay": "off"
-                        }
-                    }
-                ]
-            },
-            {
-                "type": "timeseries",
-                "id": "c3ee640f-e654-4fc7-aa2a-0dd8e9de67cb",
-                "layout": {
-                    "h": 5,
-                    "w": 12,
-                    "x": 0,
-                    "y": 37,
-                    "i": "423adbbf-8c23-45ab-b7d5-9a81b72291f1",
-                    "isResizable": true
-                },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "ntp_offset_ms{ident=~\"$ident\"}",
-                        "legend": "",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "NTP时间偏移",
-                "maxPerRow": 4,
-                "options": {
-                    "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
-                    },
-                    "legend": {
-                        "displayMode": "hidden",
-                        "placement": "bottom",
-                        "behaviour": "showItem",
-                        "selectMode": "single"
-                    },
-                    "standardOptions": {
-                        "util": "milliseconds",
-                        "decimals": 2
-                    },
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "rgba(44, 157, 61, 1)",
-                                "value": null,
-                                "type": "base"
-                            }
-                        ]
-                    },
-                    "thresholdsStyle": {
-                        "mode": "dashed"
-                    }
-                },
-                "custom": {
-                    "drawStyle": "lines",
-                    "lineInterpolation": "smooth",
-                    "spanNulls": false,
-                    "lineWidth": 2,
-                    "fillOpacity": 0.03,
-                    "gradientMode": "none",
-                    "stack": "off",
-                    "scaleDistribution": {
-                        "type": "linear"
-                    },
-                    "showPoints": "none",
-                    "pointSize": 5
-                },
-                "overrides": [
-                    {
-                        "matcher": {
-                            "id": "byFrameRefID"
-                        },
-                        "properties": {
-                            "rightYAxisDisplay": "off"
-                        }
-                    }
-                ]
-            },
-            {
-                "type": "timeseries",
-                "id": "9bb8d5ef-dc4e-419f-8e95-6dbb97b2afb6",
-                "layout": {
-                    "h": 5,
-                    "w": 12,
-                    "x": 12,
-                    "y": 37,
-                    "i": "e97f1934-26e8-4bf3-be21-95307443f146",
-                    "isResizable": true
-                },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "linux_sysctl_fs_file_nr{ident=~\"$ident\"}/linux_sysctl_fs_file_max{ident=~\"$ident\"} * 100",
-                        "legend": "",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "操作系统文件句柄使用率",
-                "description": "",
-                "maxPerRow": 4,
-                "options": {
-                    "tooltip": {
-                        "mode": "all",
-                        "sort": "desc"
-                    },
-                    "legend": {
-                        "displayMode": "hidden",
-                        "placement": "bottom",
-                        "behaviour": "showItem",
-                        "selectMode": "single"
-                    },
-                    "standardOptions": {
-                        "util": "percent",
-                        "decimals": 0
-                    },
-                    "thresholds": {
-                        "mode": "absolute",
-                        "steps": [
-                            {
-                                "color": "rgba(44, 157, 61, 1)",
-                                "value": null,
-                                "type": "base"
-                            }
-                        ]
-                    },
-                    "thresholdsStyle": {
-                        "mode": "dashed"
-                    }
-                },
-                "custom": {
-                    "drawStyle": "lines",
-                    "lineInterpolation": "smooth",
-                    "spanNulls": false,
-                    "lineWidth": 2,
-                    "fillOpacity": 0.03,
-                    "gradientMode": "none",
-                    "stack": "off",
-                    "scaleDistribution": {
-                        "type": "linear"
-                    },
-                    "showPoints": "none",
-                    "pointSize": 5
-                },
-                "overrides": [
-                    {
-                        "matcher": {
-                            "id": "byFrameRefID"
-                        },
-                        "properties": {
-                            "rightYAxisDisplay": "off"
-                        }
-                    }
-                ]
             }
         ],
         "var": [
@@ -1817,4 +1450,4 @@
         ],
         "version": "3.0.0"
     }
-}
+}

integrations/Linux/dashboards/host_table_view_demo.json

@@ -1,8 +1,6 @@
 {
-    "name": "机器台账表格视图（使用 Categraf 作为采集器）",
-    "tags": "Categraf",
-    "ident": "",
-    "uuid": 1717556327742611000,
+    "name": "机器台账表格视图",
+    "tags": "",
     "configs": {
         "links": [
             {
@@ -18,7 +16,17 @@
         ],
         "panels": [
             {
-                "type": "hexbin",
+                "custom": {
+                    "calc": "lastNotNull",
+                    "colorRange": [
+                        "thresholds"
+                    ],
+                    "detailUrl": "/built-in-components/dashboard/detail?__uuid__=1717556327744505000&ident=${__field.labels.ident}",
+                    "textMode": "valueAndName",
+                    "valueField": "Value"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
                 "id": "21b8b3ab-26aa-47cb-b814-f310f2d143aa",
                 "layout": {
                     "h": 5,
@@ -28,43 +36,18 @@
                     "x": 0,
                     "y": 0
                 },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "topk(100, cpu_usage_active{cpu=\"cpu-total\", ident=~\"$ident\"})",
-                        "instant": true,
-                        "legend": "{{ident}}",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "CPU利用率",
                 "maxPerRow": 4,
-                "custom": {
-                    "textMode": "valueAndName",
-                    "calc": "lastNotNull",
-                    "valueField": "Value",
-                    "colorRange": [
-                        "thresholds"
-                    ],
-                    "detailUrl": "/components/dashboard/detail?__uuid__=1737103014612000&ident=${__field.labels.ident}"
-                },
+                "name": "CPU利用率",
                 "options": {
+                    "standardOptions": {
+                        "util": "percent"
+                    },
                     "thresholds": {
                         "steps": [
                             {
                                 "color": "#ef3c3c",
-                                "value": 95,
-                                "type": ""
+                                "type": "",
+                                "value": 95
                             },
                             {
                                 "color": "#ff656b",
@@ -82,15 +65,38 @@
                                 "value": null
                             }
                         ]
-                    },
-                    "standardOptions": {
-                        "util": "percent",
-                        "decimals": 2
                     }
-                }
+                },
+                "targets": [
+                    {
+                        "expr": "cpu_usage_active{cpu=\"cpu-total\", ident=~\"$ident\"}",
+                        "instant": true,
+                        "legend": "{{ident}}",
+                        "maxDataPoints": 240,
+                        "refId": "A"
+                    }
+                ],
+                "transformations": [
+                    {
+                        "id": "organize",
+                        "options": {}
+                    }
+                ],
+                "type": "hexbin",
+                "version": "3.0.0"
             },
             {
-                "type": "hexbin",
+                "custom": {
+                    "calc": "lastNotNull",
+                    "colorRange": [
+                        "thresholds"
+                    ],
+                    "detailUrl": "/built-in-components/dashboard/detail?__uuid__=1717556327744505000&ident=${__field.labels.ident}",
+                    "textMode": "valueAndName",
+                    "valueField": "Value"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
                 "id": "86d4a502-21f7-4981-9b38-ed8e696b6f49",
                 "layout": {
                     "h": 5,
@@ -100,43 +106,18 @@
                     "x": 12,
                     "y": 0
                 },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "topk(100, mem_used_percent{ident=~\"$ident\"})",
-                        "instant": true,
-                        "legend": "{{ident}}",
-                        "maxDataPoints": 480,
-                        "refId": "A",
-                        "step": 15
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {}
-                    }
-                ],
-                "name": "内存利用率",
                 "maxPerRow": 4,
-                "custom": {
-                    "textMode": "valueAndName",
-                    "calc": "lastNotNull",
-                    "valueField": "Value",
-                    "colorRange": [
-                        "thresholds"
-                    ],
-                    "detailUrl": "/components/dashboard/detail?__uuid__=1737103014612000&ident=${__field.labels.ident}"
-                },
+                "name": "内存利用率",
                 "options": {
+                    "standardOptions": {
+                        "util": "percent"
+                    },
                     "thresholds": {
                         "steps": [
                             {
                                 "color": "#ef3c3c",
-                                "value": 95,
-                                "type": ""
+                                "type": "",
+                                "value": 95
                             },
                             {
                                 "color": "#ff656b",
@@ -154,15 +135,48 @@
                                 "value": null
                             }
                         ]
-                    },
-                    "standardOptions": {
-                        "util": "percent",
-                        "decimals": 2
                     }
-                }
+                },
+                "targets": [
+                    {
+                        "expr": "mem_used_percent{ident=~\"$ident\"}",
+                        "instant": true,
+                        "legend": "{{ident}}",
+                        "maxDataPoints": 240,
+                        "refId": "A"
+                    }
+                ],
+                "transformations": [
+                    {
+                        "id": "organize",
+                        "options": {}
+                    }
+                ],
+                "type": "hexbin",
+                "version": "3.0.0"
             },
             {
-                "type": "table",
+                "custom": {
+                    "aggrDimension": "ident",
+                    "calc": "lastNotNull",
+                    "colorMode": "background",
+                    "displayMode": "labelValuesToRows",
+                    "linkMode": "appendLinkColumn",
+                    "links": [
+                        {
+                            "targetBlank": true,
+                            "title": "详情",
+                            "url": "/built-in-components/dashboard/detail?__uuid__=1717556327744505000&ident=${__field.labels.ident}"
+                        }
+                    ],
+                    "nowrap": false,
+                    "showHeader": true,
+                    "sortColumn": "ident",
+                    "sortOrder": "ascend",
+                    "tableLayout": "fixed"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
                 "id": "77bf513a-8504-4d33-9efe-75aaf9abc9e4",
                 "layout": {
                     "h": 11,
@@ -172,71 +186,10 @@
                     "x": 0,
                     "y": 5
                 },
-                "version": "3.1.0",
-                "datasourceCate": "prometheus",
-                "datasourceValue": "${prom}",
-                "targets": [
-                    {
-                        "expr": "avg(cpu_usage_active{cpu=\"cpu-total\", ident=~\"$ident\"}) by (ident)",
-                        "legend": "CPU使用率",
-                        "maxDataPoints": 240,
-                        "refId": "A"
-                    },
-                    {
-                        "expr": "avg(mem_used_percent{ident=~\"$ident\"}) by (ident)",
-                        "legend": "内存使用率",
-                        "maxDataPoints": 240,
-                        "refId": "B"
-                    },
-                    {
-                        "expr": "avg(mem_total{ident=~\"$ident\"}) by (ident)",
-                        "legend": "总内存",
-                        "maxDataPoints": 240,
-                        "refId": "C"
-                    },
-                    {
-                        "expr": "avg(disk_used_percent{ident=~\"$ident\",path=\"/\"}) by (ident)",
-                        "legend": "根分区使用率",
-                        "maxDataPoints": 240,
-                        "refId": "D"
-                    }
-                ],
-                "transformations": [
-                    {
-                        "id": "organize",
-                        "options": {
-                            "renameByName": {
-                                "ident": "机器"
-                            }
-                        }
-                    }
-                ],
-                "name": "机器列表",
                 "maxPerRow": 4,
-                "custom": {
-                    "showHeader": true,
-                    "colorMode": "background",
-                    "nowrap": false,
-                    "tableLayout": "fixed",
-                    "calc": "lastNotNull",
-                    "displayMode": "labelValuesToRows",
-                    "aggrDimension": "ident",
-                    "sortColumn": "ident",
-                    "sortOrder": "ascend",
-                    "pageLimit": 500,
-                    "linkMode": "appendLinkColumn",
-                    "links": [
-                        {
-                            "targetBlank": true,
-                            "title": "详情",
-                            "url": "/components/dashboard/detail?__uuid__=1737103014612000&ident=${__field.labels.ident}"
-                        }
-                    ]
-                },
+                "name": "机器列表",
                 "options": {
-                    "standardOptions": {
-                        "decimals": 2
-                    }
+                    "standardOptions": {}
                 },
                 "overrides": [
                     {
@@ -246,8 +199,7 @@
                         },
                         "properties": {
                             "standardOptions": {
-                                "util": "percent",
-                                "decimals": 2
+                                "util": "percent"
                             },
                             "valueMappings": [
                                 {
@@ -287,8 +239,7 @@
                         },
                         "properties": {
                             "standardOptions": {
-                                "util": "percent",
-                                "decimals": 2
+                                "util": "percent"
                             },
                             "valueMappings": [
                                 {
@@ -369,32 +320,66 @@
                         },
                         "type": "special"
                     }
-                ]
+                ],
+                "targets": [
+                    {
+                        "expr": "avg(cpu_usage_active{cpu=\"cpu-total\", ident=~\"$ident\"}) by (ident)",
+                        "legend": "CPU使用率",
+                        "maxDataPoints": 240,
+                        "refId": "A"
+                    },
+                    {
+                        "expr": "avg(mem_used_percent{ident=~\"$ident\"}) by (ident)",
+                        "legend": "内存使用率",
+                        "maxDataPoints": 240,
+                        "refId": "B"
+                    },
+                    {
+                        "expr": "avg(mem_total{ident=~\"$ident\"}) by (ident)",
+                        "legend": "总内存",
+                        "maxDataPoints": 240,
+                        "refId": "C"
+                    },
+                    {
+                        "expr": "avg(disk_used_percent{ident=~\"$ident\",path=\"/\"}) by (ident)",
+                        "legend": "根分区使用率",
+                        "maxDataPoints": 240,
+                        "refId": "D"
+                    }
+                ],
+                "transformations": [
+                    {
+                        "id": "organize",
+                        "options": {
+                            "renameByName": {
+                                "ident": "机器"
+                            }
+                        }
+                    }
+                ],
+                "type": "table",
+                "version": "3.0.0"
             }
         ],
         "var": [
             {
+                "definition": "prometheus",
                 "name": "prom",
-                "label": "数据源",
-                "type": "datasource",
-                "hide": false,
-                "definition": "prometheus"
+                "type": "datasource"
             },
             {
-                "name": "ident",
-                "label": "机器",
-                "type": "query",
-                "hide": false,
-                "multi": true,
                 "allOption": true,
-                "allValue": ".*",
                 "datasource": {
                     "cate": "prometheus",
                     "value": "${prom}"
                 },
-                "definition": "label_values(system_load1,ident)"
+                "definition": "label_values(system_load1,ident)",
+                "multi": true,
+                "name": "ident",
+                "type": "query"
             }
         ],
         "version": "3.0.0"
-    }
+    },
+    "uuid": 1717556327742611000
 }

integrations/Linux/dashboards/linux_by_categraf_overview.json

@@ -0,0 +1,267 @@
+{
+    "id": 0,
+    "group_id": 0,
+    "name": "Linux Host by Categraf Overview",
+    "ident": "",
+    "tags": "",
+    "create_at": 0,
+    "create_by": "",
+    "update_at": 0,
+    "update_by": "",
+    "configs": {
+        "links": [
+            {
+                "targetBlank": true,
+                "title": "n9e",
+                "url": "https://n9e.github.io/"
+            },
+            {
+                "targetBlank": true,
+                "title": "author",
+                "url": "http://flashcat.cloud/"
+            }
+        ],
+        "panels": [
+            {
+                "collapsed": true,
+                "id": "e5d14dd7-4417-42bd-b7ba-560f34d299a2",
+                "layout": {
+                    "h": 1,
+                    "i": "e5d14dd7-4417-42bd-b7ba-560f34d299a2",
+                    "isResizable": false,
+                    "w": 24,
+                    "x": 0,
+                    "y": 0
+                },
+                "name": "整体概况",
+                "type": "row"
+            },
+            {
+                "custom": {
+                    "calc": "lastNotNull",
+                    "colSpan": 1,
+                    "colorMode": "value",
+                    "textMode": "value",
+                    "textSize": {
+                        "value": 50
+                    }
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "41f37540-e695-492a-9d2f-24bfd2d36805",
+                "layout": {
+                    "h": 3,
+                    "i": "41f37540-e695-492a-9d2f-24bfd2d36805",
+                    "isResizable": true,
+                    "w": 3,
+                    "x": 0,
+                    "y": 1
+                },
+                "name": "监控机器数",
+                "options": {
+                    "standardOptions": {}
+                },
+                "targets": [
+                    {
+                        "expr": "count(system_load1)",
+                        "refId": "A"
+                    }
+                ],
+                "type": "stat",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "baseColor": "#cd75eb",
+                    "calc": "lastNotNull",
+                    "serieWidth": 20,
+                    "sortOrder": "desc"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "585bfc50-7c92-42b1-88ee-5b725b640418",
+                "layout": {
+                    "h": 3,
+                    "i": "585bfc50-7c92-42b1-88ee-5b725b640418",
+                    "isResizable": true,
+                    "w": 9,
+                    "x": 3,
+                    "y": 1
+                },
+                "name": "内存使用率 top10",
+                "options": {
+                    "standardOptions": {},
+                    "valueMappings": [
+                        {
+                            "match": {
+                                "from": 60
+                            },
+                            "result": {
+                                "color": "#f8070e"
+                            },
+                            "type": "range"
+                        }
+                    ]
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (mem_used_percent))",
+                        "legend": "{{ident}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "barGauge",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "60b1e833-3f03-45bb-9385-a3825904a0ac",
+                "layout": {
+                    "h": 3,
+                    "i": "60b1e833-3f03-45bb-9385-a3825904a0ac",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 12,
+                    "y": 1
+                },
+                "name": "cpu使用率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "none"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (100-cpu_usage_idle{cpu=\"cpu-total\"}))",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "baseColor": "#9470ff",
+                    "calc": "lastNotNull",
+                    "serieWidth": 20,
+                    "sortOrder": "desc"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "69351db9-e646-4e5d-925a-cba29823b00d",
+                "layout": {
+                    "h": 3,
+                    "i": "69351db9-e646-4e5d-925a-cba29823b00d",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 0,
+                    "y": 4
+                },
+                "name": "磁盘分区使用率 top10",
+                "options": {
+                    "standardOptions": {},
+                    "valueMappings": [
+                        {
+                            "match": {
+                                "from": 85
+                            },
+                            "result": {
+                                "color": "#f00404"
+                            },
+                            "type": "range"
+                        }
+                    ]
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (disk_used_percent{path!~\"/var.*\"}))",
+                        "legend": "{{ident}} {{path}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "barGauge",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "e3675ed9-6d3b-4a41-8d16-d6e82037dce3",
+                "layout": {
+                    "h": 3,
+                    "i": "e3675ed9-6d3b-4a41-8d16-d6e82037dce3",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 12,
+                    "y": 4
+                },
+                "name": "设备io util top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (rate(diskio_io_time[1m])/10))",
+                        "legend": "",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            }
+        ],
+        "var": [
+            {
+                "definition": "prometheus",
+                "name": "prom",
+                "type": "datasource"
+            },
+            {
+                "datasource": {
+                    "cate": "prometheus",
+                    "value": "${prom}"
+                },
+                "definition": "label_values(system_load1,ident)",
+                "name": "ident",
+                "type": "query"
+            }
+        ],
+        "version": "3.0.0"
+    },
+    "public": 0,
+    "public_cate": 0,
+    "bgids": null,
+    "built_in": 0,
+    "hide": 0,
+    "uuid": 1717556327746983000
+}

integrations/Linux/dashboards/linux_by_exporter_overview.json

@@ -0,0 +1,269 @@
+{
+    "id": 0,
+    "group_id": 0,
+    "name": "HOST by Node Exporter Overview",
+    "ident": "",
+    "tags": "Prometheus Host",
+    "create_at": 0,
+    "create_by": "",
+    "update_at": 0,
+    "update_by": "",
+    "configs": {
+        "links": [
+            {
+                "targetBlank": true,
+                "title": "n9e",
+                "url": "https://n9e.gitee.io/"
+            },
+            {
+                "targetBlank": true,
+                "title": "author",
+                "url": "http://flashcat.cloud/"
+            }
+        ],
+        "panels": [
+            {
+                "collapsed": true,
+                "id": "3173366d-01a2-420e-8878-75124b0051b6",
+                "layout": {
+                    "h": 1,
+                    "i": "3173366d-01a2-420e-8878-75124b0051b6",
+                    "isResizable": false,
+                    "w": 24,
+                    "x": 0,
+                    "y": 0
+                },
+                "name": "整体概况",
+                "type": "row"
+            },
+            {
+                "custom": {
+                    "calc": "lastNotNull",
+                    "colSpan": 1,
+                    "colorMode": "value",
+                    "textMode": "value",
+                    "textSize": {
+                        "value": 40
+                    }
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "9a5e3292-b346-4ccf-a793-b83a2f8ac8c5",
+                "layout": {
+                    "h": 3,
+                    "i": "9a5e3292-b346-4ccf-a793-b83a2f8ac8c5",
+                    "isResizable": true,
+                    "w": 3,
+                    "x": 0,
+                    "y": 1
+                },
+                "name": "监控机器数",
+                "options": {
+                    "standardOptions": {}
+                },
+                "targets": [
+                    {
+                        "expr": "count(node_boot_time_seconds)",
+                        "refId": "A"
+                    }
+                ],
+                "type": "stat",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "description": "",
+                "id": "e1925fc8-cb05-467b-ba82-bb5cb6be7595",
+                "layout": {
+                    "h": 3,
+                    "i": "e1925fc8-cb05-467b-ba82-bb5cb6be7595",
+                    "isResizable": true,
+                    "w": 9,
+                    "x": 3,
+                    "y": 1
+                },
+                "links": [],
+                "name": "cpu使用率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10,100-(avg by (mode, instance)(rate(node_cpu_seconds_total{mode=\"idle\"}[1m])))*100)",
+                        "legend": "{{instance}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "327b7e4b-6ec1-47e1-8840-d31cf4b5532b",
+                "layout": {
+                    "h": 3,
+                    "i": "327b7e4b-6ec1-47e1-8840-d31cf4b5532b",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 12,
+                    "y": 1
+                },
+                "name": "内存使用率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10,(node_memory_MemTotal_bytes - node_memory_MemFree_bytes - (node_memory_Cached_bytes + node_memory_Buffers_bytes))/node_memory_MemTotal_bytes*100)",
+                        "legend": "{{instance}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "5a9d4a65-3f73-42cc-859e-fc0b82791b59",
+                "layout": {
+                    "h": 3,
+                    "i": "5a9d4a65-3f73-42cc-859e-fc0b82791b59",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 0,
+                    "y": 4
+                },
+                "name": "磁盘分区使用率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10,(node_filesystem_avail_bytes{device!~'rootfs', device!~\"tmpfs\",mountpoint!~\"/var/lib.*\"} * 100) / node_filesystem_size_bytes{device!~'rootfs', device!~\"tmpfs\",mountpoint!~\"/var/lib.*\"})",
+                        "legend": "{{instance}}-{{mountpoint}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "fa764e4b-5ca9-45d8-b12e-604f8743f9d9",
+                "layout": {
+                    "h": 3,
+                    "i": "fa764e4b-5ca9-45d8-b12e-604f8743f9d9",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 12,
+                    "y": 4
+                },
+                "name": "设备io util top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10,rate(node_disk_io_time_seconds_total[5m]) * 100)",
+                        "legend": "{{instance}}-{{device}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            }
+        ],
+        "var": [
+            {
+                "definition": "prometheus",
+                "name": "prom",
+                "type": "datasource"
+            },
+            {
+                "datasource": {
+                    "cate": "prometheus",
+                    "value": "${prom}"
+                },
+                "definition": "label_values(node_uname_info,  instance)",
+                "name": "node",
+                "selected": "$node",
+                "type": "query"
+            }
+        ],
+        "version": "3.0.0"
+    },
+    "public": 0,
+    "public_cate": 0,
+    "bgids": null,
+    "built_in": 0,
+    "hide": 0,
+    "uuid": 1717556327752931000
+}

integrations/Linux/dashboards/linux_by_telegraf_overview.json

@@ -0,0 +1,264 @@
+{
+    "id": 0,
+    "group_id": 0,
+    "name": "HOST by Telegraf Overview",
+    "ident": "",
+    "tags": "",
+    "create_at": 0,
+    "create_by": "",
+    "update_at": 0,
+    "update_by": "",
+    "configs": {
+        "links": [
+            {
+                "targetBlank": true,
+                "title": "n9e",
+                "url": "https://n9e.gitee.io/"
+            },
+            {
+                "targetBlank": true,
+                "title": "author",
+                "url": "http://flashcat.cloud/"
+            }
+        ],
+        "panels": [
+            {
+                "collapsed": true,
+                "id": "0f6a1394-7cf9-4958-bcfe-2fbb59e77c12",
+                "layout": {
+                    "h": 1,
+                    "i": "0f6a1394-7cf9-4958-bcfe-2fbb59e77c12",
+                    "isResizable": false,
+                    "w": 24,
+                    "x": 0,
+                    "y": 0
+                },
+                "name": "整体概况",
+                "type": "row"
+            },
+            {
+                "custom": {
+                    "calc": "lastNotNull",
+                    "colSpan": 1,
+                    "colorMode": "value",
+                    "textMode": "value",
+                    "textSize": {
+                        "value": 50
+                    }
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "877b6db5-e82c-499a-9ebc-8ad72c2891a8",
+                "layout": {
+                    "h": 3,
+                    "i": "877b6db5-e82c-499a-9ebc-8ad72c2891a8",
+                    "isResizable": true,
+                    "w": 3,
+                    "x": 0,
+                    "y": 1
+                },
+                "name": "监控机器数",
+                "options": {
+                    "standardOptions": {}
+                },
+                "targets": [
+                    {
+                        "expr": "count(system_load1)",
+                        "refId": "A"
+                    }
+                ],
+                "type": "stat",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "29a3e6ae-d278-49b3-972b-f12a6c7c091c",
+                "layout": {
+                    "h": 3,
+                    "i": "29a3e6ae-d278-49b3-972b-f12a6c7c091c",
+                    "isResizable": true,
+                    "w": 9,
+                    "x": 3,
+                    "y": 1
+                },
+                "name": "内存率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, mem_used_percent)",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "9f2a24d5-d19f-4651-b76d-add6b9011821",
+                "layout": {
+                    "h": 3,
+                    "i": "9f2a24d5-d19f-4651-b76d-add6b9011821",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 12,
+                    "y": 1
+                },
+                "name": "cpu使用率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "none"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (100-cpu_usage_idle{cpu=\"cpu-total\"}))",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "dcd60296-db84-4562-99f3-2829c2f064a4",
+                "layout": {
+                    "h": 3,
+                    "i": "dcd60296-db84-4562-99f3-2829c2f064a4",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 0,
+                    "y": 4
+                },
+                "name": "磁盘分区使用率 top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "none"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (disk_used_percent{path!~\"/var.*\"}))",
+                        "legend": "{{ident}}-{{path}}",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            },
+            {
+                "custom": {
+                    "drawStyle": "lines",
+                    "fillOpacity": 0.3,
+                    "gradientMode": "opacity",
+                    "lineInterpolation": "smooth",
+                    "lineWidth": 2,
+                    "stack": "off"
+                },
+                "datasourceCate": "prometheus",
+                "datasourceValue": "${prom}",
+                "id": "ef7df29d-7dce-4788-ae42-d21d842c67d6",
+                "layout": {
+                    "h": 3,
+                    "i": "ef7df29d-7dce-4788-ae42-d21d842c67d6",
+                    "isResizable": true,
+                    "w": 12,
+                    "x": 12,
+                    "y": 4
+                },
+                "name": "设备io util top10",
+                "options": {
+                    "legend": {
+                        "displayMode": "hidden"
+                    },
+                    "standardOptions": {},
+                    "thresholds": {},
+                    "tooltip": {
+                        "mode": "all",
+                        "sort": "desc"
+                    }
+                },
+                "targets": [
+                    {
+                        "expr": "topk(10, (rate(diskio_io_time[1m])/10))",
+                        "legend": "",
+                        "refId": "A"
+                    }
+                ],
+                "type": "timeseries",
+                "version": "2.0.0"
+            }
+        ],
+        "var": [
+            {
+                "definition": "prometheus",
+                "name": "prom",
+                "type": "datasource"
+            },
+            {
+                "datasource": {
+                    "cate": "prometheus",
+                    "value": "${prom}"
+                },
+                "definition": "label_values(system_load1,ident)",
+                "name": "ident",
+                "type": "query"
+            }
+        ],
+        "version": "3.0.0"
+    },
+    "public": 0,
+    "public_cate": 0,
+    "bgids": null,
+    "built_in": 0,
+    "hide": 0,
+    "uuid": 1717556327757522000
+}

integrations/Linux/markdown/README.md

@@ -55,7 +55,6 @@ nr_alloc_batch = 0
 
 统计 ARP 包的数量，该插件依赖 cgo，如果需要该插件需要下载 `with-cgo` 的 categraf 发布包。
 
-
 ## ntp
 
-监控机器时间偏移量，只需要给出 ntp 服务端地址，Categraf 就会周期性去请求，对比本机时间，得到偏移量，监控指标是 ntp_offset_ms 顾名思义，单位是毫秒，一般这个值不能超过 1000
+监控机器时间偏移量，只需要给出 ntp 服务端地址，Categraf 就会周期性去请求，对比本机时间，得到偏移量，监控指标是 ntp_offset_ms 顾名思义，单位是毫秒，一般这个值不能超过 1000

memsto/event_processor_cache.go

@@ -20,7 +20,7 @@ type EventProcessorCacheType struct {
 	stats           *Stats
 
 	sync.RWMutex
-	eventPipelines map[int64]*models.EventPipeline // key: pipeline id
+	processors map[int64]*models.EventPipeline // key: pipeline id
 }
 
 func NewEventProcessorCache(ctx *ctx.Context, stats *Stats) *EventProcessorCacheType {
@@ -29,7 +29,7 @@ func NewEventProcessorCache(ctx *ctx.Context, stats *Stats) *EventProcessorCache
 		statLastUpdated: -1,
 		ctx:             ctx,
 		stats:           stats,
-		eventPipelines:  make(map[int64]*models.EventPipeline),
+		processors:      make(map[int64]*models.EventPipeline),
 	}
 	epc.SyncEventProcessors()
 	return epc
@@ -41,7 +41,7 @@ func (epc *EventProcessorCacheType) Reset() {
 
 	epc.statTotal = -1
 	epc.statLastUpdated = -1
-	epc.eventPipelines = make(map[int64]*models.EventPipeline)
+	epc.processors = make(map[int64]*models.EventPipeline)
 }
 
 func (epc *EventProcessorCacheType) StatChanged(total, lastUpdated int64) bool {
@@ -54,7 +54,7 @@ func (epc *EventProcessorCacheType) StatChanged(total, lastUpdated int64) bool {
 
 func (epc *EventProcessorCacheType) Set(m map[int64]*models.EventPipeline, total, lastUpdated int64) {
 	epc.Lock()
-	epc.eventPipelines = m
+	epc.processors = m
 	epc.Unlock()
 
 	// only one goroutine used, so no need lock
@@ -65,29 +65,17 @@ func (epc *EventProcessorCacheType) Set(m map[int64]*models.EventPipeline, total
 func (epc *EventProcessorCacheType) Get(processorId int64) *models.EventPipeline {
 	epc.RLock()
 	defer epc.RUnlock()
-	return epc.eventPipelines[processorId]
-}
-
-func (epc *EventProcessorCacheType) GetProcessorsById(processorId int64) []models.Processor {
-	epc.RLock()
-	defer epc.RUnlock()
-
-	eventPipeline, ok := epc.eventPipelines[processorId]
-	if !ok {
-		return []models.Processor{}
-	}
-
-	return eventPipeline.Processors
+	return epc.processors[processorId]
 }
 
 func (epc *EventProcessorCacheType) GetProcessorIds() []int64 {
 	epc.RLock()
 	defer epc.RUnlock()
 
-	count := len(epc.eventPipelines)
+	count := len(epc.processors)
 	list := make([]int64, 0, count)
-	for eid := range epc.eventPipelines {
-		list = append(list, eid)
+	for processorId := range epc.processors {
+		list = append(list, processorId)
 	}
 
 	return list
@@ -137,18 +125,7 @@ func (epc *EventProcessorCacheType) syncEventProcessors() error {
 
 	m := make(map[int64]*models.EventPipeline)
 	for i := 0; i < len(lst); i++ {
-		eventPipeline := lst[i]
-		for _, p := range eventPipeline.ProcessorConfigs {
-			processor, err := models.GetProcessorByType(p.Typ, p.Config)
-			if err != nil {
-				logger.Warningf("event_pipeline_id: %d, event:%+v, processor:%+v get processor err: %+v", eventPipeline.ID, eventPipeline, p, err)
-				continue
-			}
-
-			eventPipeline.Processors = append(eventPipeline.Processors, processor)
-		}
-
-		m[lst[i].ID] = eventPipeline
+		m[lst[i].ID] = lst[i]
 	}
 
 	epc.Set(m, stat.Total, stat.LastUpdated)

models/alert_aggr_view.go

@@ -37,35 +37,33 @@ func (v *AlertAggrView) Verify() error {
 		return errors.New("rule is blank")
 	}
 
-	if !strings.Contains(v.Rule, "{{") {
-		var validFields = []string{
-			"cluster",
-			"group_id",
-			"group_name",
-			"rule_id",
-			"rule_name",
-			"severity",
-			"runbook_url",
-			"target_ident",
-			"target_note",
+	var validFields = []string{
+		"cluster",
+		"group_id",
+		"group_name",
+		"rule_id",
+		"rule_name",
+		"severity",
+		"runbook_url",
+		"target_ident",
+		"target_note",
+	}
+
+	arr := strings.Split(v.Rule, "::")
+	for i := 0; i < len(arr); i++ {
+		pair := strings.Split(arr[i], ":")
+		if len(pair) != 2 {
+			return errors.New("rule invalid")
 		}
 
-		arr := strings.Split(v.Rule, "::")
-		for i := 0; i < len(arr); i++ {
-			pair := strings.Split(arr[i], ":")
-			if len(pair) != 2 {
-				return errors.New("rule invalid")
-			}
+		if !(pair[0] == "field" || pair[0] == "tagkey") {
+			return errors.New("rule invalid")
+		}
 
-			if !(pair[0] == "field" || pair[0] == "tagkey") {
-				return errors.New("rule invalid")
-			}
-
-			if pair[0] == "field" {
-				// 只支持有限的field
-				if !slice.ContainsString(validFields, pair[1]) {
-					return fmt.Errorf("unsupported field: %s", pair[1])
-				}
+		if pair[0] == "field" {
+			// 只支持有限的field
+			if !slice.ContainsString(validFields, pair[1]) {
+				return fmt.Errorf("unsupported field: %s", pair[1])
 			}
 		}
 	}
@@ -139,14 +137,3 @@ func AlertAggrViewGet(ctx *ctx.Context, where string, args ...interface{}) (*Ale
 
 	return lst[0], nil
 }
-
-func GetAlertAggrViewByViewID(ctx *ctx.Context, viewID int64) (*AlertAggrView, error) {
-	view, err := AlertAggrViewGet(ctx, "id = ?", viewID)
-	if err != nil {
-		return nil, err
-	}
-	if view == nil {
-		return nil, errors.New("alert aggr view not found")
-	}
-	return view, nil
-}

models/alert_cur_event.go

@@ -4,7 +4,6 @@ import (
 	"bytes"
 	"encoding/json"
 	"fmt"
-	"net/http"
 	"reflect"
 	"strconv"
 	"strings"
@@ -16,7 +15,6 @@ import (
 	"github.com/ccfos/nightingale/v6/pkg/tplx"
 	"github.com/ccfos/nightingale/v6/pkg/unit"
 
-	"github.com/toolkits/pkg/ginx"
 	"github.com/toolkits/pkg/logger"
 )
 
@@ -76,15 +74,7 @@ type AlertCurEvent struct {
 	RecoverConfig      RecoverConfig       `json:"recover_config" gorm:"-"`
 	RuleHash           string              `json:"rule_hash" gorm:"-"`
 	ExtraInfoMap       []map[string]string `json:"extra_info_map" gorm:"-"`
-	NotifyRuleIds      []int64             `json:"notify_rule_ids" gorm:"serializer:json"`
-
-	NotifyVersion int                `json:"notify_version"  gorm:"-"` // 0: old, 1: new
-	NotifyRules   []*EventNotifyRule `json:"notify_rules" gorm:"-"`
-}
-
-type EventNotifyRule struct {
-	Id   int64  `json:"id"`
-	Name string `json:"name"`
+	NotifyRuleIDs      []int64             `json:"notify_rule_ids" gorm:"-"`
 }
 
 func (e *AlertCurEvent) SetTagsMap() {
@@ -298,49 +288,7 @@ func (e *AlertCurEvent) ParseURL(url string) (string, error) {
 	return body.String(), nil
 }
 
-func parseAggrRules(rule string) []*AggrRule {
-	aggrRules := strings.Split(rule, "::") // e.g. field:group_name::field:severity::tagkey:ident
-
-	if len(aggrRules) == 0 {
-		ginx.Bomb(http.StatusBadRequest, "rule empty")
-	}
-
-	rules := make([]*AggrRule, len(aggrRules))
-	for i := 0; i < len(aggrRules); i++ {
-		pair := strings.Split(aggrRules[i], ":")
-		if len(pair) != 2 {
-			ginx.Bomb(http.StatusBadRequest, "rule invalid")
-		}
-
-		if !(pair[0] == "field" || pair[0] == "tagkey") {
-			ginx.Bomb(http.StatusBadRequest, "rule invalid")
-		}
-
-		rules[i] = &AggrRule{
-			Type:  pair[0],
-			Value: pair[1],
-		}
-	}
-	return rules
-}
-
-func (e *AlertCurEvent) GenCardTitle(rule string) (string, error) {
-	if strings.Contains(rule, "{{") {
-		// 有 {{ 表示使用的是新的配置方式，使用 go template 进行格式化
-
-		tmpl, err := template.New("card_title").Parse(rule)
-		if err != nil {
-			return fmt.Sprintf("failed to parse card title: %v", err), nil
-		}
-
-		var buf bytes.Buffer
-		if err := tmpl.Execute(&buf, e); err != nil {
-			return fmt.Sprintf("failed to execute card title: %v", err), nil
-		}
-		return buf.String(), nil
-	}
-
-	rules := parseAggrRules(rule)
+func (e *AlertCurEvent) GenCardTitle(rules []*AggrRule) string {
 	arr := make([]string, len(rules))
 	for i := 0; i < len(rules); i++ {
 		rule := rules[i]
@@ -354,10 +302,10 @@ func (e *AlertCurEvent) GenCardTitle(rule string) (string, error) {
 		}
 
 		if len(arr[i]) == 0 {
-			arr[i] = "Others"
+			arr[i] = "Null"
 		}
 	}
-	return strings.Join(arr, "::"), nil
+	return strings.Join(arr, "::")
 }
 
 func (e *AlertCurEvent) GetTagValue(tagkey string) string {
@@ -445,7 +393,6 @@ func (e *AlertCurEvent) ToHis(ctx *ctx.Context) *AlertHisEvent {
 		LastEvalTime:     e.LastEvalTime,
 		NotifyCurNumber:  e.NotifyCurNumber,
 		FirstTriggerTime: e.FirstTriggerTime,
-		NotifyRuleIds:    e.NotifyRuleIds,
 	}
 }
 
@@ -461,22 +408,6 @@ func (e *AlertCurEvent) DB2FE() error {
 	if err := json.Unmarshal([]byte(e.RuleConfig), &e.RuleConfigJson); err != nil {
 		return err
 	}
-
-	e.TagsMap = make(map[string]string)
-	for i := 0; i < len(e.TagsJSON); i++ {
-		pair := strings.TrimSpace(e.TagsJSON[i])
-		if pair == "" {
-			continue
-		}
-
-		arr := strings.SplitN(pair, "=", 2)
-		if len(arr) != 2 {
-			continue
-		}
-
-		e.TagsMap[arr[0]] = arr[1]
-	}
-
 	return nil
 }
 
@@ -594,7 +525,7 @@ func (e *AlertCurEvent) FillNotifyGroups(ctx *ctx.Context, cache map[int64]*User
 }
 
 func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime, etime int64,
-	severity []int64, dsIds []int64, cates []string, ruleId int64, query string, eventIds []int64) (int64, error) {
+	severity int, dsIds []int64, cates []string, ruleId int64, query string) (int64, error) {
 	session := DB(ctx).Model(&AlertCurEvent{})
 	if stime != 0 && etime != 0 {
 		session = session.Where("trigger_time between ? and ?", stime, etime)
@@ -607,8 +538,8 @@ func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime,
 		session = session.Where("group_id in ?", bgids)
 	}
 
-	if len(severity) > 0 {
-		session = session.Where("severity in ?", severity)
+	if severity >= 0 {
+		session = session.Where("severity = ?", severity)
 	}
 
 	if len(dsIds) > 0 {
@@ -623,9 +554,6 @@ func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime,
 		session = session.Where("rule_id = ?", ruleId)
 	}
 
-	if len(eventIds) > 0 {
-		session = session.Where("id in ?", eventIds)
-	}
 	if query != "" {
 		arr := strings.Fields(query)
 		for i := 0; i < len(arr); i++ {
@@ -638,10 +566,9 @@ func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime,
 }
 
 func AlertCurEventsGet(ctx *ctx.Context, prods []string, bgids []int64, stime, etime int64,
-	severity []int64, dsIds []int64, cates []string, ruleId int64, query string, limit, offset int, eventIds []int64) (
+	severity int, dsIds []int64, cates []string, ruleId int64, query string, limit, offset int) (
 	[]AlertCurEvent, error) {
 	session := DB(ctx).Model(&AlertCurEvent{})
-
 	if stime != 0 && etime != 0 {
 		session = session.Where("trigger_time between ? and ?", stime, etime)
 	}
@@ -653,8 +580,8 @@ func AlertCurEventsGet(ctx *ctx.Context, prods []string, bgids []int64, stime, e
 		session = session.Where("group_id in ?", bgids)
 	}
 
-	if len(severity) > 0 {
-		session = session.Where("severity in ?", severity)
+	if severity >= 0 {
+		session = session.Where("severity = ?", severity)
 	}
 
 	if len(dsIds) > 0 {
@@ -669,10 +596,6 @@ func AlertCurEventsGet(ctx *ctx.Context, prods []string, bgids []int64, stime, e
 		session = session.Where("rule_id = ?", ruleId)
 	}
 
-	if len(eventIds) > 0 {
-		session = session.Where("id in ?", eventIds)
-	}
-
 	if query != "" {
 		arr := strings.Fields(query)
 		for i := 0; i < len(arr); i++ {
@@ -1033,9 +956,9 @@ func (e *AlertCurEvent) DeepCopy() *AlertCurEvent {
 		}
 	}
 
-	if e.NotifyRuleIds != nil {
-		eventCopy.NotifyRuleIds = make([]int64, len(e.NotifyRuleIds))
-		copy(eventCopy.NotifyRuleIds, e.NotifyRuleIds)
+	if e.NotifyRuleIDs != nil {
+		eventCopy.NotifyRuleIDs = make([]int64, len(e.NotifyRuleIDs))
+		copy(eventCopy.NotifyRuleIDs, e.NotifyRuleIDs)
 	}
 
 	eventCopy.RuleConfigJson = e.RuleConfigJson

models/alert_his_event.go

@@ -55,10 +55,6 @@ type AlertHisEvent struct {
 	NotifyCurNumber    int               `json:"notify_cur_number"`    // notify: current number
 	FirstTriggerTime   int64             `json:"first_trigger_time"`   // 连续告警的首次告警时间
 	ExtraConfig        interface{}       `json:"extra_config" gorm:"-"`
-	NotifyRuleIds      []int64           `json:"notify_rule_ids" gorm:"serializer:json"`
-
-	NotifyVersion int                `json:"notify_version" gorm:"-"`
-	NotifyRules   []*EventNotifyRule `json:"notify_rules" gorm:"-"`
 }
 
 func (e *AlertHisEvent) TableName() string {
@@ -243,15 +239,6 @@ func AlertHisEventGetById(ctx *ctx.Context, id int64) (*AlertHisEvent, error) {
 	return AlertHisEventGet(ctx, "id=?", id)
 }
 
-func AlertHisEventBatchDelete(ctx *ctx.Context, timestamp int64, severities []int, limit int) (int64, error) {
-	db := DB(ctx).Where("last_eval_time < ?", timestamp)
-	if len(severities) > 0 {
-		db = db.Where("severity IN (?)", severities)
-	}
-	res := db.Limit(limit).Delete(&AlertHisEvent{})
-	return res.RowsAffected, res.Error
-}
-
 func (m *AlertHisEvent) UpdateFieldsMap(ctx *ctx.Context, fields map[string]interface{}) error {
 	return DB(ctx).Model(m).Updates(fields).Error
 }

models/alert_rule.go

@@ -28,8 +28,6 @@ const (
 	PROMETHEUS    = "prometheus"
 	TDENGINE      = "tdengine"
 	ELASTICSEARCH = "elasticsearch"
-	MYSQL         = "mysql"
-	POSTGRESQL    = "pgsql"
 
 	CLICKHOUSE = "ck"
 )
@@ -507,7 +505,7 @@ func (ar *AlertRule) Verify() error {
 	ar.AppendTags = strings.TrimSpace(ar.AppendTags)
 	arr := strings.Fields(ar.AppendTags)
 	for i := 0; i < len(arr); i++ {
-		if !strings.Contains(arr[i], "=") {
+		if len(strings.Split(arr[i], "=")) != 2 {
 			return fmt.Errorf("AppendTags(%s) invalid", arr[i])
 		}
 	}
@@ -840,6 +838,7 @@ func (ar *AlertRule) FillNotifyGroups(ctx *ctx.Context, cache map[int64]*UserGro
 }
 
 func (ar *AlertRule) FE2DB() error {
+
 	if len(ar.EnableStimesJSON) > 0 {
 		ar.EnableStime = strings.Join(ar.EnableStimesJSON, " ")
 		ar.EnableEtime = strings.Join(ar.EnableEtimesJSON, " ")
@@ -867,16 +866,7 @@ func (ar *AlertRule) FE2DB() error {
 	ar.NotifyChannels = strings.Join(ar.NotifyChannelsJSON, " ")
 	ar.NotifyGroups = strings.Join(ar.NotifyGroupsJSON, " ")
 	ar.Callbacks = strings.Join(ar.CallbacksJSON, " ")
-
-	for i := range ar.AppendTagsJSON {
-		// 后面要把多个标签拼接在一起，所以每个标签里不能有空格
-		ar.AppendTagsJSON[i] = strings.ReplaceAll(ar.AppendTagsJSON[i], " ", "")
-	}
-
-	if len(ar.AppendTagsJSON) > 0 {
-		ar.AppendTags = strings.Join(ar.AppendTagsJSON, " ")
-	}
-
+	ar.AppendTags = strings.Join(ar.AppendTagsJSON, " ")
 	algoParamsByte, err := json.Marshal(ar.AlgoParamsJson)
 	if err != nil {
 		return fmt.Errorf("marshal algo_params err:%v", err)
@@ -1185,21 +1175,12 @@ func (ar *AlertRule) IsLokiRule() bool {
 	return ar.Prod == LOKI || ar.Cate == LOKI
 }
 
-func (ar *AlertRule) IsTdengineRule() bool {
-	return ar.Cate == TDENGINE
-}
-
 func (ar *AlertRule) IsHostRule() bool {
 	return ar.Prod == HOST
 }
 
-func (ar *AlertRule) IsInnerRule() bool {
-	return ar.Cate == TDENGINE ||
-		ar.Cate == CLICKHOUSE ||
-		ar.Cate == ELASTICSEARCH ||
-		ar.Prod == LOKI || ar.Cate == LOKI ||
-		ar.Cate == MYSQL ||
-		ar.Cate == POSTGRESQL
+func (ar *AlertRule) IsTdengineRule() bool {
+	return ar.Cate == TDENGINE
 }
 
 func (ar *AlertRule) GetRuleType() string {

models/alert_subscribe.go

@@ -116,10 +116,6 @@ func (s *AlertSubscribe) Verify() error {
 		return errors.New("severities is required")
 	}
 
-	if len(s.NotifyRuleIds) > 0 {
-		return nil
-	}
-
 	if s.UserGroupIds != "" && s.NewChannels == "" {
 		// 如果指定了用户组，那么新告警的通知渠道必须指定，否则容易出现告警规则中没有指定通知渠道，导致订阅通知时，没有通知渠道
 		return errors.New("new_channels is required")
@@ -412,9 +408,9 @@ func (s *AlertSubscribe) ModifyEvent(event *AlertCurEvent) {
 	}
 
 	if len(s.NotifyRuleIds) > 0 {
-		event.NotifyRuleIds = s.NotifyRuleIds
+		event.NotifyRuleIDs = s.NotifyRuleIds
 	} else {
-		event.NotifyRuleIds = []int64{}
+		event.NotifyRuleIDs = []int64{}
 	}
 
 	event.NotifyGroups = s.UserGroupIds

models/datasource.go

@@ -170,31 +170,6 @@ func DatasourceGet(ctx *ctx.Context, id int64) (*Datasource, error) {
 	return ds, ds.DB2FE()
 }
 
-type DatasourceInfo struct {
-	Id         int64  `json:"id"`
-	Name       string `json:"name"`
-	PluginType string `json:"plugin_type"`
-}
-
-func GetDatasourceInfosByIds(ctx *ctx.Context, ids []int64) ([]*DatasourceInfo, error) {
-	if len(ids) == 0 {
-		return []*DatasourceInfo{}, nil
-	}
-
-	var dsInfos []*DatasourceInfo
-	err := DB(ctx).
-		Model(&Datasource{}).
-		Select("id", "name", "plugin_type").
-		Where("id in ?", ids).
-		Find(&dsInfos).Error
-
-	if err != nil {
-		return nil, err
-	}
-
-	return dsInfos, nil
-}
-
 func (ds *Datasource) Get(ctx *ctx.Context) error {
 	err := DB(ctx).Where("id = ?", ds.Id).First(ds).Error
 	if err != nil {

models/embedded_product.go

@@ -1,147 +0,0 @@
-package models
-
-import (
-	"encoding/json"
-	"time"
-
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-
-	"github.com/pkg/errors"
-	"github.com/toolkits/pkg/logger"
-	"github.com/toolkits/pkg/str"
-	"gorm.io/gorm"
-	"gorm.io/gorm/clause"
-)
-
-type EmbeddedProduct struct {
-	ID        int64   `json:"id" gorm:"primaryKey"` // 主键
-	Name      string  `json:"name" gorm:"column:name;type:varchar(255)"`
-	URL       string  `json:"url" gorm:"column:url;type:varchar(255)"`
-	IsPrivate bool    `json:"is_private" gorm:"column:is_private;type:boolean"`
-	TeamIDs   []int64 `json:"team_ids" gorm:"serializer:json"`
-	CreateAt  int64   `json:"create_at" gorm:"column:create_at;not null;default:0"`
-	CreateBy  string  `json:"create_by" gorm:"column:create_by;type:varchar(64);not null;default:''"`
-	UpdateAt  int64   `json:"update_at" gorm:"column:update_at;not null;default:0"`
-	UpdateBy  string  `json:"update_by" gorm:"column:update_by;type:varchar(64);not null;default:''"`
-}
-
-func (e *EmbeddedProduct) TableName() string {
-	return "embedded_product"
-}
-
-func (e *EmbeddedProduct) AfterFind(tx *gorm.DB) (err error) {
-	if e.TeamIDs == nil {
-		e.TeamIDs = []int64{}
-	}
-	return nil
-}
-
-func (e *EmbeddedProduct) Verify() error {
-	if e.Name == "" {
-		return errors.New("Name is blank")
-	}
-
-	if str.Dangerous(e.Name) {
-		return errors.New("Name has invalid characters")
-	}
-
-	if e.URL == "" {
-		return errors.New("URL is blank")
-	}
-
-	if e.IsPrivate && len(e.TeamIDs) == 0 {
-		return errors.New("TeamIDs is blank")
-	}
-
-	return nil
-}
-
-func AddEmbeddedProduct(ctx *ctx.Context, eps []EmbeddedProduct) error {
-	now := time.Now().Unix()
-
-	for i := range eps {
-		if err := eps[i].Verify(); err != nil {
-			return errors.Wrapf(err, "invalid entry %v", eps[i])
-		}
-		eps[i].CreateAt = now
-		eps[i].UpdateAt = now
-	}
-
-	// 用主键做冲突判断，有冲突则更新（UPSERT）
-	return DB(ctx).Clauses(clause.OnConflict{
-		UpdateAll: true, // 冲突时更新所有字段
-	}).Create(&eps).Error
-}
-
-func EmbeddedProductGets(ctx *ctx.Context) ([]*EmbeddedProduct, error) {
-	var list []*EmbeddedProduct
-	err := DB(ctx).Find(&list).Error
-	return list, err
-}
-
-func GetEmbeddedProductByID(ctx *ctx.Context, id int64) (*EmbeddedProduct, error) {
-	var ep EmbeddedProduct
-	err := DB(ctx).Where("id = ?", id).First(&ep).Error
-	return &ep, err
-}
-
-func UpdateEmbeddedProduct(ctx *ctx.Context, ep *EmbeddedProduct) error {
-	if err := ep.Verify(); err != nil {
-		return err
-	}
-	return DB(ctx).Save(ep).Error
-}
-
-func DeleteEmbeddedProduct(ctx *ctx.Context, id int64) error {
-	return DB(ctx).Where("id = ?", id).Delete(&EmbeddedProduct{}).Error
-}
-
-func CanMigrateEP(ctx *ctx.Context) bool {
-	var count int64
-	err := DB(ctx).Model(&EmbeddedProduct{}).Count(&count).Error
-	if err != nil {
-		logger.Errorf("failed to get embedded-product table count, err:%v", err)
-		return false
-	}
-	return count <= 0
-}
-
-func MigrateEP(ctx *ctx.Context) {
-	var lst []string
-	_ = DB(ctx).Model(&Configs{}).Where("ckey=?  and external=? ", "embedded-dashboards", 0).Pluck("cval", &lst).Error
-	if len(lst) > 0 {
-		var oldData []DashboardConfig
-		if err := json.Unmarshal([]byte(lst[0]), &oldData); err != nil {
-			return
-		}
-
-		if len(oldData) < 1 {
-			return
-		}
-
-		now := time.Now().Unix()
-		var newData []EmbeddedProduct
-		for _, v := range oldData {
-			newData = append(newData, EmbeddedProduct{
-				Name:      v.Name,
-				URL:       v.URL,
-				IsPrivate: false,
-				TeamIDs:   []int64{},
-				CreateBy:  "system",
-				CreateAt:  now,
-				UpdateAt:  now,
-				UpdateBy:  "system",
-			})
-		}
-		err := DB(ctx).Create(&newData).Error
-		if err != nil {
-			logger.Errorf("failed to create embedded-product, err:%v", err)
-		}
-	}
-}
-
-type DashboardConfig struct {
-	ID   string `json:"id"`
-	Name string `json:"name"`
-	URL  string `json:"url"`
-}

models/event_pipeline.go

@@ -11,24 +11,22 @@ import (
 
 // EventPipeline 事件Pipeline模型
 type EventPipeline struct {
-	ID               int64             `json:"id" gorm:"primaryKey"`
-	Name             string            `json:"name" gorm:"type:varchar(128)"`
-	TeamIds          []int64           `json:"team_ids" gorm:"type:text;serializer:json"`
-	TeamNames        []string          `json:"team_names" gorm:"-"`
-	Description      string            `json:"description" gorm:"type:varchar(255)"`
-	FilterEnable     bool              `json:"filter_enable" gorm:"type:bigint"`
-	LabelFilters     []TagFilter       `json:"label_filters" gorm:"type:text;serializer:json"`
-	AttrFilters      []TagFilter       `json:"attribute_filters" gorm:"type:text;serializer:json"`
-	ProcessorConfigs []ProcessorConfig `json:"processors" gorm:"type:text;serializer:json"`
-	CreateAt         int64             `json:"create_at" gorm:"type:bigint"`
-	CreateBy         string            `json:"create_by" gorm:"type:varchar(64)"`
-	UpdateAt         int64             `json:"update_at" gorm:"type:bigint"`
-	UpdateBy         string            `json:"update_by" gorm:"type:varchar(64)"`
-
-	Processors []Processor `json:"-" gorm:"-"`
+	ID           int64       `json:"id" gorm:"primaryKey"`
+	Name         string      `json:"name" gorm:"type:varchar(128)"`
+	TeamIds      []int64     `json:"team_ids" gorm:"type:text;serializer:json"`
+	TeamNames    []string    `json:"team_names" gorm:"-"`
+	Description  string      `json:"description" gorm:"type:varchar(255)"`
+	FilterEnable bool        `json:"filter_enable" gorm:"type:tinyint(1)"`
+	LabelFilters []TagFilter `json:"label_filters" gorm:"type:text;serializer:json"`
+	AttrFilters  []TagFilter `json:"attribute_filters" gorm:"type:text;serializer:json"`
+	Processors   []Processor `json:"processors" gorm:"type:text;serializer:json"`
+	CreateAt     int64       `json:"create_at" gorm:"type:bigint"`
+	CreateBy     string      `json:"create_by" gorm:"type:varchar(64)"`
+	UpdateAt     int64       `json:"update_at" gorm:"type:bigint"`
+	UpdateBy     string      `json:"update_by" gorm:"type:varchar(64)"`
 }
 
-type ProcessorConfig struct {
+type Processor struct {
 	Typ    string      `json:"typ"`
 	Config interface{} `json:"config"`
 }
@@ -46,20 +44,22 @@ func (e *EventPipeline) Verify() error {
 		return errors.New("team_ids cannot be empty")
 	}
 
-	if len(e.TeamIds) == 0 {
+	return nil
+}
+
+func (e *EventPipeline) DB2FE() {
+	if e.TeamIds == nil {
 		e.TeamIds = make([]int64, 0)
 	}
-	if len(e.LabelFilters) == 0 {
+	if e.LabelFilters == nil {
 		e.LabelFilters = make([]TagFilter, 0)
 	}
-	if len(e.AttrFilters) == 0 {
+	if e.AttrFilters == nil {
 		e.AttrFilters = make([]TagFilter, 0)
 	}
-	if len(e.ProcessorConfigs) == 0 {
-		e.ProcessorConfigs = make([]ProcessorConfig, 0)
+	if e.Processors == nil {
+		e.Processors = make([]Processor, 0)
 	}
-
-	return nil
 }
 
 // CreateEventPipeline 创建事件Pipeline
@@ -74,7 +74,7 @@ func GetEventPipeline(ctx *ctx.Context, id int64) (*EventPipeline, error) {
 	if err != nil {
 		return nil, err
 	}
-	pipeline.Verify()
+	pipeline.DB2FE()
 	return &pipeline, nil
 }
 
@@ -102,13 +102,13 @@ func ListEventPipelines(ctx *ctx.Context) ([]*EventPipeline, error) {
 	}
 
 	var pipelines []*EventPipeline
-	err := DB(ctx).Order("name asc").Find(&pipelines).Error
+	err := DB(ctx).Order("name desc").Find(&pipelines).Error
 	if err != nil {
 		return nil, err
 	}
 
 	for _, p := range pipelines {
-		p.Verify()
+		p.DB2FE()
 	}
 
 	return pipelines, nil

models/message_tpl.go

@@ -505,15 +505,15 @@ var NewTplMap = map[string]string{
 {{if $event.IsRecovered}}恢复时间：{{timeformat $event.LastEvalTime}}{{else}}触发时间: {{timeformat $event.TriggerTime}}   
 触发时值: {{$event.TriggerValue}}{{end}}   
 发送时间: {{timestamp}}`,
-	Telegram: `<b>级别状态: {{if $event.IsRecovered}}💚 S{{$event.Severity}} Recovered{{else}}⚠️ S{{$event.Severity}} Triggered{{end}}</b>
-<b>规则标题</b>: {{$event.RuleName}}{{if $event.RuleNote}}   
-<b>规则备注</b>: {{$event.RuleNote}}{{end}}{{if $event.TargetIdent}}   
-<b>监控对象</b>: {{$event.TargetIdent}}{{end}}   
-<b>监控指标</b>: {{$event.TagsJSON}}{{if not $event.IsRecovered}}   
-<b>触发时值</b>: {{$event.TriggerValue}}{{end}}   
-{{if $event.IsRecovered}}<b>恢复时间</b>: {{timeformat $event.LastEvalTime}}{{else}}<b>首次触发时间</b>: {{timeformat $event.FirstTriggerTime}}{{end}}   
-{{$time_duration := sub now.Unix $event.FirstTriggerTime }}{{if $event.IsRecovered}}{{$time_duration = sub $event.LastEvalTime $event.FirstTriggerTime }}{{end}}<b>距离首次告警</b>: {{humanizeDurationInterface $time_duration}}
-<b>发送时间</b>: {{timestamp}}`,
+	Telegram: `**级别状态**: {{if $event.IsRecovered}}<font color="info">S{{$event.Severity}} Recovered</font>{{else}}<font color="warning">S{{$event.Severity}} Triggered</font>{{end}}   
+**规则标题**: {{$event.RuleName}}{{if $event.RuleNote}}   
+**规则备注**: {{$event.RuleNote}}{{end}}{{if $event.TargetIdent}}   
+**监控对象**: {{$event.TargetIdent}}{{end}}   
+**监控指标**: {{$event.TagsJSON}}{{if not $event.IsRecovered}}   
+**触发时值**: {{$event.TriggerValue}}{{end}}   
+{{if $event.IsRecovered}}**恢复时间**: {{timeformat $event.LastEvalTime}}{{else}}**首次触发时间**: {{timeformat $event.FirstTriggerTime}}{{end}}   
+{{$time_duration := sub now.Unix $event.FirstTriggerTime }}{{if $event.IsRecovered}}{{$time_duration = sub $event.LastEvalTime $event.FirstTriggerTime }}{{end}}**距离首次告警**: {{humanizeDurationInterface $time_duration}}
+**发送时间**: {{timestamp}}`,
 	Wecom: `**级别状态**: {{if $event.IsRecovered}}<font color="info">💚S{{$event.Severity}} Recovered</font>{{else}}<font color="warning">💔S{{$event.Severity}} Triggered</font>{{end}}       
 **规则标题**: {{$event.RuleName}}{{if $event.RuleNote}}   
 **规则备注**: {{$event.RuleNote}}{{end}}{{if $event.TargetIdent}}   

models/migrate/migrate.go

@@ -68,7 +68,7 @@ func MigrateTables(db *gorm.DB) error {
 		&Board{}, &BoardBusigroup{}, &Users{}, &SsoConfig{}, &models.BuiltinMetric{},
 		&models.MetricFilter{}, &models.NotificaitonRecord{}, &models.TargetBusiGroup{},
 		&models.UserToken{}, &models.DashAnnotation{}, MessageTemplate{}, NotifyRule{}, NotifyChannelConfig{}, &EsIndexPatternMigrate{},
-		&models.EventPipeline{}, &models.EmbeddedProduct{}, &models.SourceToken{}}
+		&models.EventPipeline{}}
 
 	if isPostgres(db) {
 		dts = append(dts, &models.PostgresBuiltinComponent{})
@@ -80,20 +80,24 @@ func MigrateTables(db *gorm.DB) error {
 		dts = append(dts, &imodels.TaskSchedulerHealth{})
 	}
 
-	asyncDts := []interface{}{&AlertHisEvent{}, &AlertCurEvent{}}
-	go func() {
-		defer func() {
-			if r := recover(); r != nil {
-				logger.Errorf("panic to migrate table: %v", r)
+	if !columnHasIndex(db, &AlertHisEvent{}, "original_tags") ||
+		!columnHasIndex(db, &AlertCurEvent{}, "original_tags") {
+		asyncDts := []interface{}{&AlertHisEvent{}, &AlertCurEvent{}}
+
+		go func() {
+			defer func() {
+				if r := recover(); r != nil {
+					logger.Errorf("panic to migrate table: %v", r)
+				}
+			}()
+
+			for _, dt := range asyncDts {
+				if err := db.AutoMigrate(dt); err != nil {
+					logger.Errorf("failed to migrate table %+v err:%v", dt, err)
+				}
 			}
 		}()
-
-		for _, dt := range asyncDts {
-			if err := db.AutoMigrate(dt); err != nil {
-				logger.Errorf("failed to migrate table %+v err:%v", dt, err)
-			}
-		}
-	}()
+	}
 
 	if !db.Migrator().HasTable(&models.BuiltinPayload{}) {
 		dts = append(dts, &models.BuiltinPayload{})
@@ -174,6 +178,11 @@ func InsertPermPoints(db *gorm.DB) {
 		Operation: "/help/variable-configs",
 	})
 
+	ops = append(ops, models.RoleOperation{
+		RoleName:  "Admin",
+		Operation: "/permissions",
+	})
+
 	ops = append(ops, models.RoleOperation{
 		RoleName:  "Standard",
 		Operation: "/ibex-settings",
@@ -242,8 +251,9 @@ func InsertPermPoints(db *gorm.DB) {
 	for _, op := range ops {
 		var count int64
 
-		session := db.Session(&gorm.Session{}).Model(&models.RoleOperation{})
-		err := session.Where("operation = ? AND role_name = ?", op.Operation, op.RoleName).Count(&count).Error
+		err := db.Model(&models.RoleOperation{}).
+			Where("operation = ? AND role_name = ?", op.Operation, op.RoleName).
+			Count(&count).Error
 
 		if err != nil {
 			logger.Errorf("check role operation exists failed, %v", err)
@@ -254,7 +264,7 @@ func InsertPermPoints(db *gorm.DB) {
 			continue
 		}
 
-		err = session.Create(&op).Error
+		err = db.Model(&models.RoleOperation{}).Create(&op).Error
 		if err != nil {
 			logger.Errorf("insert role operation failed, %v", err)
 		}
@@ -302,14 +312,12 @@ type TaskRecord struct {
 	EventId int64 `gorm:"column:event_id;bigint(20);not null;default:0;comment:event id;index:idx_event_id"`
 }
 type AlertHisEvent struct {
-	LastEvalTime  int64   `gorm:"column:last_eval_time;bigint(20);not null;default:0;comment:for time filter;index:idx_last_eval_time"`
-	OriginalTags  string  `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
-	NotifyRuleIds []int64 `gorm:"column:notify_rule_ids;type:text;serializer:json;comment:notify rule ids"`
+	LastEvalTime int64  `gorm:"column:last_eval_time;bigint(20);not null;default:0;comment:for time filter;index:idx_last_eval_time"`
+	OriginalTags string `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
 }
 
 type AlertCurEvent struct {
-	OriginalTags  string  `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
-	NotifyRuleIds []int64 `gorm:"column:notify_rule_ids;type:text;serializer:json;comment:notify rule ids"`
+	OriginalTags string `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
 }
 
 type Target struct {

models/migrate/migrate_test.go

@@ -34,6 +34,11 @@ func TestInsertPermPoints(t *testing.T) {
 		Operation: "/help/variable-configs",
 	})
 
+	ops = append(ops, models.RoleOperation{
+		RoleName:  "Admin",
+		Operation: "/permissions",
+	})
+
 	ops = append(ops, models.RoleOperation{
 		RoleName:  "Standard",
 		Operation: "/ibex-settings",

models/notify_channel.go

@@ -157,7 +157,7 @@ func (ncc *NotifyChannelConfig) SendScript(events []*AlertCurEvent, tpl map[stri
 		return "", "", fmt.Errorf("script or path is empty")
 	}
 
-	fpath := ".notify_script_" + strconv.FormatInt(ncc.ID, 10)
+	fpath := ".notify_scriptt"
 	if config.Path != "" {
 		fpath = config.Path
 	} else {
@@ -803,7 +803,6 @@ func (ncc *NotifyChannelConfig) SendEmailNow(events []*AlertCurEvent, tpl map[st
 	s, err := d.Dial()
 	if err != nil {
 		logger.Errorf("email_sender: failed to dial: %s", err)
-		return err
 	}
 
 	m := gomail.NewMessage()
@@ -1236,12 +1235,12 @@ var NotiChMap = []*NotifyChannelConfig{
 		Name: "Telegram", Ident: Telegram, RequestType: "http", Weight: 7, Enable: true,
 		RequestConfig: &RequestConfig{
 			HTTPRequestConfig: &HTTPRequestConfig{
-				URL:    "https://api.telegram.org/bot{{$params.token}}/sendMessage",
-				Method: "POST", Headers: map[string]string{"Content-Type": "application/json"},
+				URL:     "https://api.telegram.org/bot{{$params.token}}/sendMessage",
+				Method:  "POST",
 				Timeout: 10000, Concurrency: 5, RetryTimes: 3, RetryInterval: 100,
 				Request: RequestDetail{
 					Parameters: map[string]string{"chat_id": "{{$params.chat_id}}"},
-					Body:       `{"text":"{{$tpl.content}}","parse_mode": "HTML"}`,
+					Body:       `{"parse_mode": "markdown", "text": "{{$tpl.content}}"}`,
 				},
 			},
 		},

models/notify_rule.go

@@ -216,7 +216,7 @@ func NotifyRulesGet(ctx *ctx.Context, where string, args ...interface{}) ([]*Not
 	if where != "" && len(args) > 0 {
 		session = session.Where(where, args...)
 	}
-	err := session.Order("name asc").Find(&lst).Error
+	err := session.Find(&lst).Error
 	if err != nil {
 		return nil, err
 	}

models/prom_alert_rule.go

@@ -2,7 +2,6 @@ package models
 
 import (
 	"fmt"
-	"strings"
 	"time"
 
 	"github.com/toolkits/pkg/logger"
@@ -41,7 +40,7 @@ func ConvertAlert(rule PromRule, interval string, datasouceQueries []DatasourceQ
 	if len(rule.Labels) > 0 {
 		for k, v := range rule.Labels {
 			if k != "severity" {
-				appendTags = append(appendTags, fmt.Sprintf("%s=%s", strings.ReplaceAll(k, " ", ""), strings.ReplaceAll(v, " ", "")))
+				appendTags = append(appendTags, fmt.Sprintf("%s=%s", k, v))
 			} else {
 				switch v {
 				case "critical":

models/source_token.go

@@ -1,48 +0,0 @@
-package models
-
-import (
-	"time"
-
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
-)
-
-type SourceToken struct {
-	Id         int64  `json:"id" gorm:"primaryKey"`
-	SourceType string `json:"source_type" gorm:"column:source_type;type:varchar(64);not null;default:''"`
-	SourceId   string `json:"source_id" gorm:"column:source_id;type:varchar(255);not null;default:''"`
-	Token      string `json:"token" gorm:"column:token;type:varchar(255);not null;default:''"`
-	ExpireAt   int64  `json:"expire_at" gorm:"type:bigint;not null;default:0"`
-	CreateAt   int64  `json:"create_at" gorm:"type:bigint;not null;default:0"`
-	CreateBy   string `json:"create_by" gorm:"type:varchar(64);not null;default:''"`
-}
-
-func (SourceToken) TableName() string {
-	return "source_token"
-}
-
-func (st *SourceToken) Add(ctx *ctx.Context) error {
-	return Insert(ctx, st)
-}
-
-// GetSourceTokenBySource 根据源类型和源ID获取源令牌
-func GetSourceTokenBySource(ctx *ctx.Context, sourceType, sourceId, token string) (*SourceToken, error) {
-	var st SourceToken
-	err := DB(ctx).Where("source_type = ? AND source_id = ? AND token = ?", sourceType, sourceId, token).First(&st).Error
-	if err != nil {
-		return nil, err
-	}
-	return &st, nil
-}
-
-func (st *SourceToken) IsExpired() bool {
-	if st.ExpireAt == 0 {
-		return false // 0 表示永不过期
-	}
-	return time.Now().Unix() > st.ExpireAt
-}
-
-func CleanupExpiredTokens(ctx *ctx.Context) (int64, error) {
-	now := time.Now().Unix()
-	result := DB(ctx).Where("expire_at > 0 AND expire_at < ?", now).Delete(&SourceToken{})
-	return result.RowsAffected, result.Error
-}