refactor: optimize event processor

Co-authored-by: Haobo Zhang <43698160+haobo8@users.noreply.github.com>

README.md

@@ -3,7 +3,7 @@
     <img src="doc/img/Nightingale_L_V.png" alt="nightingale - cloud native monitoring" width="100" /></a>
 </p>
 <p align="center">
-  <b>开源告警管理专家 一体化的可观测平台</b>
+  <b>开源告警管理专家</b>
 </p>
 
 <p align="center">
@@ -27,77 +27,86 @@
 
 [English](./README_en.md) | [中文](./README.md)
 
-## 夜莺 Nightingale 是什么
+## 夜莺是什么
 
-> 夜莺 Nightingale 是什么，解决什么问题？以大家都很熟悉的 Grafana 做个类比，Grafana 擅长对接各种各样的数据源，然后提供灵活、强大、好看的可视化面板。夜莺则擅长对接各种多样的数据源，提供灵活、强大、高效的监控告警管理能力。从发展路径和定位来说，夜莺和 Grafana 很像，可以总结为一句话：可视化就用 Grafana，监控告警就找夜莺。
->
-> 在可视化领域，Grafana 是毫无争议的领导者，Grafana 在影响力、装机量、用户群、开发者数量等各个维度的数字上，相比夜莺都是追赶的榜样。巨无霸往往都是从一个切入点打开局面的，Grafana Labs 有了在可视化领域 Grafana 这个王牌，逐步扩展到整个可观测性方向，比如 Logging 维度有 Loki，Tracing 维度有 Tempo，Profiling 维度有收购来的 Pyroscope，On-call 维度有同样是收购来的 Grafana-OnCall 项目，还有时序数据库 Mimir、eBPF 采集器 Beyla、OpenTelemetry 采集器 Alloy、前端监控 SDK Faro，最终构成了一个完整的可观测性工具矩阵，但整个飞轮都是从 Grafana 项目开始转动起来的。
->
->夜莺，则是从监控告警这个切入点打开局面，也逐步横向做了相应扩展，比如夜莺也自研了可视化面板，如果你想有一个 all-in-one 的监控告警+可视化的工具，那么用夜莺也是正确的选择；比如 OnCall 方向，夜莺可以和 [Flashduty SaaS](https://flashcat.cloud/product/flashcat-duty/) 服务无缝的集成；在采集器方向，夜莺有配套的 [Categraf](https://flashcat.cloud/product/categraf)，可以一个采集器中管理所有的 exporter，并同时支持指标和日志的采集，极大减轻工程师维护的采集器数量和工作量（这个点太痛了，你可能也遇到过业务团队吐槽采集器数量比业务应用进程数量还多的窘况吧）。
+夜莺监控（Nightingale）是一款侧重告警的监控类开源项目。类似 Grafana 的数据源集成方式，夜莺也是对接多种既有的数据源，不过 Grafana 侧重在可视化，夜莺是侧重在告警引擎、告警事件的处理和分发。
 
-夜莺 Nightingale 作为一款开源云原生监控工具，最初由滴滴开发和开源，并于 2022 年 5 月 11 日，捐赠予中国计算机学会开源发展委员会（CCF ODC），为 CCF ODC 成立后接受捐赠的第一个开源项目。在 GitHub 上有超过 10000 颗星，是广受关注和使用的开源监控工具。夜莺的核心研发团队，也是 Open-Falcon 项目原核心研发人员，从 2014 年（Open-Falcon 是 2014 年开源）算起来，也有 10 年了，只为把监控做到极致。
+夜莺监控项目，最初由滴滴开发和开源，并于 2022 年 5 月 11 日，捐赠予中国计算机学会开源发展委员会（CCF ODC），为 CCF ODC 成立后接受捐赠的第一个开源项目。
 
+## 夜莺的工作逻辑
 
-## 快速开始
-- 👉 [文档中心](https://flashcat.cloud/docs/) | [下载中心](https://flashcat.cloud/download/nightingale/)
-- ❤️ [报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml)
-- ℹ️ 为了提供更快速的访问体验，上述文档和下载站点托管于 [FlashcatCloud](https://flashcat.cloud)
-- 💡 前后端代码分离，前端代码仓库：[https://github.com/n9e/fe](https://github.com/n9e/fe)
+很多用户已经自行采集了指标、日志数据，此时就把存储库（VictoriaMetrics、ElasticSearch等）作为数据源接入夜莺，即可在夜莺里配置告警规则、通知规则，完成告警事件的生成和派发。
 
-## 功能特点
+![夜莺产品架构](doc/img/readme/20240221152601.png)
 
-- 对接多种时序库，实现统一监控告警管理：支持对接的时序库包括 Prometheus、VictoriaMetrics、Thanos、Mimir、M3DB、TDengine 等。
-- 对接日志库，实现针对日志的监控告警：支持对接的日志库包括 ElasticSearch、Loki 等。
-- 专业告警能力：内置支持多种告警规则，可以扩展支持常见通知媒介，支持告警屏蔽/抑制/订阅/自愈、告警事件管理。
-- 高性能可视化引擎：支持多种图表样式，内置众多 Dashboard 模版，也可导入 Grafana 模版，开箱即用，开源协议商业友好。
-- 支持常见采集器：支持 [Categraf](https://flashcat.cloud/product/categraf)、Telegraf、Grafana-agent、Datadog-agent、各种 Exporter 作为采集器，没有什么数据是不能监控的。
-- 👀无缝搭配 [Flashduty](https://flashcat.cloud/product/flashcat-duty/)：实现告警聚合收敛、认领、升级、排班、IM集成，确保告警处理不遗漏，减少打扰，高效协同。
+夜莺项目本身不提供监控数据采集能力。推荐您使用 [Categraf](https://github.com/flashcatcloud/categraf) 作为采集器，可以和夜莺丝滑对接。
 
+[Categraf](https://github.com/flashcatcloud/categraf) 可以采集操作系统、网络设备、各类中间件、数据库的监控数据，通过 Remote Write 协议推送给夜莺，夜莺把监控数据转存到时序库（如 Prometheus、VictoriaMetrics 等），并提供告警和可视化能力。
 
-## 截图演示
-
-
-你可以在页面的右上角，切换语言和主题，目前我们支持英语、简体中文、繁体中文。
-
-![语言切换](doc/img/readme/n9e-switch-i18n.png)
-
-即时查询，类似 Prometheus 内置的查询分析页面，做 ad-hoc 查询，夜莺做了一些 UI 优化，同时提供了一些内置 promql 指标，让不太了解 promql 的用户也可以快速查询。
-
-![即时查询](doc/img/readme/20240513103305.png)
-
-当然，也可以直接通过指标视图查看，有了指标视图，即时查询基本可以不用了，或者只有高端玩家使用即时查询，普通用户直接通过指标视图查询即可。
-
-![指标视图](doc/img/readme/20240513103530.png)
-
-夜莺内置了常用仪表盘，可以直接导入使用。也可以导入 Grafana 仪表盘，不过只能兼容 Grafana 基本图表，如果已经习惯了 Grafana 建议继续使用 Grafana 看图，把夜莺作为一个告警引擎使用。
-
-![内置仪表盘](doc/img/readme/20240513103628.png)
-
-除了内置的仪表盘，也内置了很多告警规则，开箱即用。
-
-![内置告警规则](doc/img/readme/20240513103825.png)
-
-
-
-## 产品架构
-
-社区使用夜莺最多的场景就是使用夜莺做告警引擎，对接多套时序库，统一告警规则管理。绘图仍然使用 Grafana 居多。作为一个告警引擎，夜莺的产品架构如下：
-
-![产品架构](doc/img/readme/20240221152601.png)
-
-对于个别边缘机房，如果和中心夜莺服务端网络链路不好，希望提升告警可用性，我们也提供边缘机房告警引擎下沉部署模式，这个模式下，即便网络割裂，告警功能也不受影响。
+对于个别边缘机房，如果和中心夜莺服务端网络链路不好，希望提升告警可用性，夜莺也提供边缘机房告警引擎下沉部署模式，这个模式下，即便边缘和中心端网络割裂，告警功能也不受影响。
 
 ![边缘部署模式](doc/img/readme/20240222102119.png)
 
+> 上图中，机房A和中心机房的网络链路很好，所以直接由中心端的夜莺进程做告警引擎，机房B和中心机房的网络链路不好，所以在机房B部署了 `n9e-edge` 做告警引擎，对机房B的数据源做告警判定。
 
-## 交流渠道
-- 报告Bug，优先推荐提交[夜莺GitHub Issue](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=kind%2Fbug&projects=&template=bug_report.yml)
-- 关注[这个公众号](https://gitlink.org.cn/UlricQin)了解更多夜莺动态和知识
-- 加我微信：`picobyte`（我已关闭好友验证）拉入微信群，备注：`夜莺互助群`
+## 告警降噪、升级、协同
+
+夜莺的侧重点是做告警引擎，即负责产生告警事件，并根据规则做灵活派发，内置支持 20 种通知媒介（电话、短信、邮件、钉钉、飞书、企微、Slack 等）。
+
+如果您有更高级的需求，比如：
+
+- 想要把公司的多套监控系统产生的事件聚拢到一个平台，统一做收敛降噪、响应处理、数据分析
+- 想要支持人员的排班，践行 On-call 文化，想要支持告警认领、升级（避免遗漏）、协同处理
+
+那夜莺是不合适的，推荐您选用 [FlashDuty](https://flashcat.cloud/product/flashcat-duty/) 这样的 On-call 产品，产品简单易用，也有免费套餐。
+
+
+## 相关资料 & 交流渠道
+- 📚 [夜莺介绍PPT](https://mp.weixin.qq.com/s/Mkwx_46xrltSq8NLqAIYow) 对您了解夜莺各项关键特性会有帮助（PPT链接在文末）
+- 👉 [文档中心](https://flashcat.cloud/docs/) 为了更快的访问速度，站点托管在 [FlashcatCloud](https://flashcat.cloud)
+- ❤️ [报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml) 写清楚问题描述、复现步骤、截图等信息，更容易得到答案
+- 💡 前后端代码分离，前端代码仓库：[https://github.com/n9e/fe](https://github.com/n9e/fe)
+- 🎯 关注[这个公众号](https://gitlink.org.cn/UlricQin)了解更多夜莺动态和知识
+- 🌟 加我微信：`picobyte`（我已关闭好友验证）拉入微信群，备注：`夜莺互助群`，如果已经把夜莺上到生产环境，可联系我拉入资深监控用户群
+
+
+## 关键特性简介
+
+![夜莺告警规则](doc/img/readme/2025-05-23_18-43-37.png)
+
+- 夜莺支持告警规则、屏蔽规则、订阅规则、通知规则，内置支持 20 种通知媒介，支持消息模板自定义
+- 支持事件管道，对告警事件做 Pipeline 处理，方便和自有系统做自动化整合，比如给告警事件附加一些元信息，对事件做 relabel
+- 支持业务组概念，引入权限体系，分门别类管理各类规则
+- 很多数据库、中间件内置了告警规则，可以直接导入使用，也可以直接导入 Prometheus 的告警规则
+- 支持告警自愈，即告警之后自动触发一个脚本执行一些预定义的逻辑，比如清理一下磁盘、抓一下现场等
+
+![夜莺事件大盘](doc/img/readme/2025-05-30_08-49-28.png)
+
+- 夜莺存档了历史告警事件，支持多维度的查询和统计
+- 支持灵活的聚合分组，一目了然看到公司的告警事件分布情况
+
+![夜莺集成中心](doc/img/readme/2025-05-23_18-46-06.png)
+
+- 夜莺内置常用操作系统、中间件、数据库的的指标说明、仪表盘、告警规则，不过都是社区贡献的，整体也是参差不齐
+- 夜莺直接接收 Remote Write、OpenTSDB、Datadog、Falcon 等多种协议的数据，故而可以和各类 Agent 对接
+- 夜莺支持 Prometheus、ElasticSearch、Loki、TDEngine 等多种数据源，可以对其中的数据做告警
+- 夜莺可以很方便内嵌企业内部系统，比如 Grafana、CMDB 等，甚至可以配置这些内嵌系统的菜单可见性
+
+
+![夜莺仪表盘](doc/img/readme/2025-05-23_18-49-02.png)
+
+- 夜莺支持仪表盘功能，支持常见的图表类型，也内置了一些仪表盘，上图是其中一个仪表盘的截图。
+- 如果你已经习惯了 Grafana，建议仍然使用 Grafana 看图。Grafana 在看图方面道行更深。
+- 机器相关的监控数据，如果是 Categraf 采集的，建议使用夜莺自带的仪表盘查看，因为 Categraf 的指标命名 Follow 的是 Telegraf 的命名方式，和 Node Exporter 不同
+- 因为夜莺有个业务组的概念，机器可以归属不同的业务组，有时在仪表盘里只想查看当前所属业务组的机器，所以夜莺的仪表盘可以和业务组联动
 
 ## 广受关注
 [![Stargazers over time](https://api.star-history.com/svg?repos=ccfos/nightingale&type=Date)](https://star-history.com/#ccfos/nightingale&Date)
 
+## 感谢众多企业的信赖
+
+![夜莺客户](doc/img/readme/logos.png)
+
 ## 社区共建
 - ❇️ 请阅读浏览[夜莺开源项目和社区治理架构草案](./doc/community-governance.md)，真诚欢迎每一位用户、开发者、公司以及组织，使用夜莺监控、积极反馈 Bug、提交功能需求、分享最佳实践，共建专业、活跃的夜莺开源社区。
 - ❤️ 夜莺贡献者

alert/astats/stats.go

@@ -25,6 +25,7 @@ type Stats struct {
 	CounterHeartbeatErrorTotal  *prometheus.CounterVec
 	CounterSubEventTotal        *prometheus.CounterVec
 	GaugeQuerySeriesCount       *prometheus.GaugeVec
+	GaugeRuleEvalDuration       *prometheus.GaugeVec
 	GaugeNotifyRecordQueueSize  prometheus.Gauge
 }
 
@@ -136,6 +137,13 @@ func NewSyncStats() *Stats {
 		Help:      "The size of notify record queue.",
 	})
 
+	GaugeRuleEvalDuration := prometheus.NewGaugeVec(prometheus.GaugeOpts{
+		Namespace: namespace,
+		Subsystem: subsystem,
+		Name:      "rule_eval_duration_ms",
+		Help:      "Duration of rule eval in milliseconds.",
+	}, []string{"rule_id", "datasource_id"})
+
 	CounterVarFillingQuery := prometheus.NewCounterVec(prometheus.CounterOpts{
 		Namespace: namespace,
 		Subsystem: subsystem,
@@ -158,6 +166,7 @@ func NewSyncStats() *Stats {
 		CounterHeartbeatErrorTotal,
 		CounterSubEventTotal,
 		GaugeQuerySeriesCount,
+		GaugeRuleEvalDuration,
 		GaugeNotifyRecordQueueSize,
 		CounterVarFillingQuery,
 	)
@@ -177,6 +186,7 @@ func NewSyncStats() *Stats {
 		CounterHeartbeatErrorTotal:  CounterHeartbeatErrorTotal,
 		CounterSubEventTotal:        CounterSubEventTotal,
 		GaugeQuerySeriesCount:       GaugeQuerySeriesCount,
+		GaugeRuleEvalDuration:       GaugeRuleEvalDuration,
 		GaugeNotifyRecordQueueSize:  GaugeNotifyRecordQueueSize,
 		CounterVarFillingQuery:      CounterVarFillingQuery,
 	}

alert/dispatch/dispatch.go

@@ -80,6 +80,8 @@ func NewDispatch(alertRuleCache *memsto.AlertRuleCacheType, userCache *memsto.Us
 		ctx:    ctx,
 		Astats: astats,
 	}
+
+	pipeline.Init()
 	return notify
 }
 
@@ -146,8 +148,8 @@ func (e *Dispatch) reloadTpls() error {
 
 func (e *Dispatch) HandleEventWithNotifyRule(eventOrigin *models.AlertCurEvent) {
 
-	if len(eventOrigin.NotifyRuleIDs) > 0 {
-		for _, notifyRuleId := range eventOrigin.NotifyRuleIDs {
+	if len(eventOrigin.NotifyRuleIds) > 0 {
+		for _, notifyRuleId := range eventOrigin.NotifyRuleIds {
 			// 深拷贝新的 event，避免并发修改 event 冲突
 			eventCopy := eventOrigin.DeepCopy()
 
@@ -161,7 +163,7 @@ func (e *Dispatch) HandleEventWithNotifyRule(eventOrigin *models.AlertCurEvent)
 				continue
 			}
 
-			var processors []pipeline.Processor
+			var processors []models.Processor
 			for _, pipelineConfig := range notifyRule.PipelineConfigs {
 				if !pipelineConfig.Enable {
 					continue
@@ -178,26 +180,24 @@ func (e *Dispatch) HandleEventWithNotifyRule(eventOrigin *models.AlertCurEvent)
 					continue
 				}
 
-				for _, p := range eventPipeline.Processors {
-					processor, err := pipeline.GetProcessorByType(p.Typ, p.Config)
-					if err != nil {
-						logger.Warningf("notify_id: %d, event:%+v, processor:%+v type not found", notifyRuleId, eventCopy, p)
-						continue
-					}
-					processors = append(processors, processor)
-				}
+				processors = append(processors, e.eventProcessorCache.GetProcessorsById(pipelineConfig.PipelineId)...)
 			}
 
 			for _, processor := range processors {
 				logger.Infof("before processor notify_id: %d, event:%+v, processor:%+v", notifyRuleId, eventCopy, processor)
-				processor.Process(e.ctx, eventCopy)
-				logger.Infof("after processor notify_id: %d, event:%+v, processor:%+v", notifyRuleId, eventCopy, processor)
+				eventCopy, res, err := processor.Process(e.ctx, eventCopy)
+				logger.Infof("after processor notify_id: %d, event:%+v, processor:%+v, res:%v, err:%v", notifyRuleId, eventCopy, processor, res, err)
 				if eventCopy == nil {
 					logger.Warningf("notify_id: %d, event:%+v, processor:%+v, event is nil", notifyRuleId, eventCopy, processor)
 					break
 				}
 			}
 
+			if eventCopy == nil {
+				// 如果 eventCopy 为 nil，说明 eventCopy 被 processor drop 掉了, 不再发送通知
+				continue
+			}
+
 			// notify
 			for i := range notifyRule.NotifyConfigs {
 				if !NotifyRuleApplicable(&notifyRule.NotifyConfigs[i], eventCopy) {
@@ -442,6 +442,10 @@ func (e *Dispatch) sendV2(events []*models.AlertCurEvent, notifyRuleId int64, no
 
 	switch notifyChannel.RequestType {
 	case "flashduty":
+		if len(flashDutyChannelIDs) == 0 {
+			flashDutyChannelIDs = []int64{0} // 如果 flashduty 通道没有配置，则使用 0, 给 SendFlashDuty 判断使用, 不给 flashduty 传 channel_id 参数
+		}
+
 		for i := range flashDutyChannelIDs {
 			respBody, err := notifyChannel.SendFlashDuty(events, flashDutyChannelIDs[i], e.notifyChannelCache.GetHttpClient(notifyChannel.ID))
 			logger.Infof("notify_id: %d, channel_name: %v, event:%+v, IntegrationUrl: %v dutychannel_id: %v, respBody: %v, err: %v", notifyRuleId, notifyChannel.Name, events[0], notifyChannel.RequestConfig.FlashDutyRequestConfig.IntegrationUrl, flashDutyChannelIDs[i], respBody, err)
@@ -499,11 +503,6 @@ func (e *Dispatch) HandleEventNotify(event *models.AlertCurEvent, isSubscribe bo
 		return
 	}
 
-	if e.blockEventNotify(rule, event) {
-		logger.Infof("block event notify: rule_id:%d event:%+v", rule.Id, event)
-		return
-	}
-
 	fillUsers(event, e.userCache, e.userGroupCache)
 
 	var (
@@ -540,25 +539,6 @@ func (e *Dispatch) HandleEventNotify(event *models.AlertCurEvent, isSubscribe bo
 	}
 }
 
-func (e *Dispatch) blockEventNotify(rule *models.AlertRule, event *models.AlertCurEvent) bool {
-	ruleType := rule.GetRuleType()
-
-	// 若为机器则先看机器是否删除
-	if ruleType == models.HOST {
-		host, ok := e.targetCache.Get(event.TagsMap["ident"])
-		if !ok || host == nil {
-			return true
-		}
-	}
-
-	// 恢复通知，检测规则配置是否改变
-	// if event.IsRecovered && event.RuleHash != rule.Hash() {
-	// 	return true
-	// }
-
-	return false
-}
-
 func (e *Dispatch) handleSubs(event *models.AlertCurEvent) {
 	// handle alert subscribes
 	subscribes := make([]*models.AlertSubscribe, 0)
@@ -728,6 +708,11 @@ func (e *Dispatch) HandleIbex(rule *models.AlertRule, event *models.AlertCurEven
 	}
 	json.Unmarshal([]byte(rule.RuleConfig), &ruleConfig)
 
+	if event.IsRecovered {
+		// 恢复事件不需要走故障自愈的逻辑
+		return
+	}
+
 	for _, t := range ruleConfig.TaskTpls {
 		if t.TplId == 0 {
 			continue

alert/dispatch/log.go

@@ -24,7 +24,7 @@ func LogEvent(event *models.AlertCurEvent, location string, err ...error) {
 		location,
 		event.RuleId,
 		event.SubRuleId,
-		event.NotifyRuleIDs,
+		event.NotifyRuleIds,
 		event.Cluster,
 		event.TagsJSON,
 		event.TriggerValue,

alert/eval/alert_rule.go

@@ -93,7 +93,7 @@ func (s *Scheduler) syncAlertRules() {
 		}
 
 		ruleType := rule.GetRuleType()
-		if rule.IsPrometheusRule() || rule.IsLokiRule() || rule.IsTdengineRule() || rule.IsClickHouseRule() || rule.IsElasticSearch() {
+		if rule.IsPrometheusRule() || rule.IsInnerRule() {
 			datasourceIds := s.datasourceCache.GetIDsByDsCateAndQueries(rule.Cate, rule.DatasourceQueries)
 			for _, dsId := range datasourceIds {
 				if !naming.DatasourceHashRing.IsHit(strconv.FormatInt(dsId, 10), fmt.Sprintf("%d", rule.Id), s.aconf.Heartbeat.Endpoint) {

alert/eval/eval.go

@@ -172,7 +172,7 @@ func (arw *AlertRuleWorker) Eval() {
 	case models.LOKI:
 		anomalyPoints, err = arw.GetPromAnomalyPoint(cachedRule.RuleConfig)
 	default:
-		anomalyPoints, recoverPoints = arw.GetAnomalyPoint(cachedRule, arw.Processor.DatasourceId())
+		anomalyPoints, recoverPoints, err = arw.GetAnomalyPoint(cachedRule, arw.Processor.DatasourceId())
 	}
 
 	if err != nil {
@@ -232,6 +232,10 @@ func (arw *AlertRuleWorker) Stop() {
 
 func (arw *AlertRuleWorker) GetPromAnomalyPoint(ruleConfig string) ([]models.AnomalyPoint, error) {
 	var lst []models.AnomalyPoint
+	start := time.Now()
+	defer func() {
+		arw.Processor.Stats.GaugeRuleEvalDuration.WithLabelValues(fmt.Sprintf("%v", arw.Rule.Id), fmt.Sprintf("%v", arw.Processor.DatasourceId())).Set(float64(time.Since(start).Milliseconds()))
+	}()
 
 	var rule *models.PromRuleConfig
 	if err := json.Unmarshal([]byte(ruleConfig), &rule); err != nil {
@@ -742,6 +746,10 @@ func combine(paramKeys []string, paraMap map[string][]string, index int, current
 
 func (arw *AlertRuleWorker) GetHostAnomalyPoint(ruleConfig string) ([]models.AnomalyPoint, error) {
 	var lst []models.AnomalyPoint
+	start := time.Now()
+	defer func() {
+		arw.Processor.Stats.GaugeRuleEvalDuration.WithLabelValues(fmt.Sprintf("%v", arw.Rule.Id), fmt.Sprintf("%v", arw.Processor.DatasourceId())).Set(float64(time.Since(start).Milliseconds()))
+	}()
 
 	var rule *models.HostRuleConfig
 	if err := json.Unmarshal([]byte(ruleConfig), &rule); err != nil {
@@ -1414,13 +1422,18 @@ func fillVar(curRealQuery string, paramKey string, val string) string {
 	return curRealQuery
 }
 
-func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64) ([]models.AnomalyPoint, []models.AnomalyPoint) {
+func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64) ([]models.AnomalyPoint, []models.AnomalyPoint, error) {
 	// 获取查询和规则判断条件
+	start := time.Now()
+	defer func() {
+		arw.Processor.Stats.GaugeRuleEvalDuration.WithLabelValues(fmt.Sprintf("%v", arw.Rule.Id), fmt.Sprintf("%v", arw.Processor.DatasourceId())).Set(float64(time.Since(start).Milliseconds()))
+	}()
+
 	points := []models.AnomalyPoint{}
 	recoverPoints := []models.AnomalyPoint{}
 	ruleConfig := strings.TrimSpace(rule.RuleConfig)
 	if ruleConfig == "" {
-		logger.Warningf("rule_eval:%d promql is blank", rule.Id)
+		logger.Warningf("rule_eval:%d ruleConfig is blank", rule.Id)
 		arw.Processor.Stats.CounterRuleEvalErrorTotal.WithLabelValues(fmt.Sprintf("%v", arw.Processor.DatasourceId()), GET_RULE_CONFIG, arw.Processor.BusiGroupCache.GetNameByBusiGroupId(arw.Rule.GroupId), fmt.Sprintf("%v", arw.Rule.Id)).Inc()
 		arw.Processor.Stats.GaugeQuerySeriesCount.WithLabelValues(
 			fmt.Sprintf("%v", arw.Rule.Id),
@@ -1428,7 +1441,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 			"",
 		).Set(0)
 
-		return points, recoverPoints
+		return points, recoverPoints, fmt.Errorf("rule_eval:%d ruleConfig is blank", rule.Id)
 	}
 
 	var ruleQuery models.RuleQuery
@@ -1436,7 +1449,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 	if err != nil {
 		logger.Warningf("rule_eval:%d promql parse error:%s", rule.Id, err.Error())
 		arw.Processor.Stats.CounterRuleEvalErrorTotal.WithLabelValues(fmt.Sprintf("%v", arw.Processor.DatasourceId()), GET_RULE_CONFIG, arw.Processor.BusiGroupCache.GetNameByBusiGroupId(arw.Rule.GroupId), fmt.Sprintf("%v", arw.Rule.Id)).Inc()
-		return points, recoverPoints
+		return points, recoverPoints, fmt.Errorf("rule_eval:%d promql parse error:%s", rule.Id, err.Error())
 	}
 
 	arw.Inhibit = ruleQuery.Inhibit
@@ -1456,7 +1469,8 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 					fmt.Sprintf("%v", arw.Processor.DatasourceId()),
 					fmt.Sprintf("%v", i),
 				).Set(-2)
-				continue
+
+				return points, recoverPoints, fmt.Errorf("rule_eval:%d datasource:%d not exists", rule.Id, dsId)
 			}
 
 			ctx := context.WithValue(context.Background(), "delay", int64(rule.Delay))
@@ -1471,7 +1485,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 					fmt.Sprintf("%v", i),
 				).Set(-1)
 
-				continue
+				return points, recoverPoints, fmt.Errorf("rule_eval:%d query data error: %v", rule.Id, err)
 			}
 
 			arw.Processor.Stats.GaugeQuerySeriesCount.WithLabelValues(
@@ -1505,6 +1519,7 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 		for _, query := range ruleQuery.Queries {
 			ref, unit, err := GetQueryRefAndUnit(query)
 			if err != nil {
+				logger.Warningf("rule_eval rid:%d query:%+v get ref and unit error:%s", rule.Id, query, err.Error())
 				continue
 			}
 			unitMap[ref] = unit
@@ -1671,5 +1686,5 @@ func (arw *AlertRuleWorker) GetAnomalyPoint(rule *models.AlertRule, dsId int64)
 		}
 	}
 
-	return points, recoverPoints
+	return points, recoverPoints, nil
 }

alert/mute/mute.go

@@ -44,6 +44,12 @@ func TimeSpanMuteStrategy(rule *models.AlertRule, event *models.AlertCurEvent) b
 	triggerTime := tm.Format("15:04")
 	triggerWeek := strconv.Itoa(int(tm.Weekday()))
 
+	if rule.EnableDaysOfWeek == "" {
+		// 如果规则没有配置生效时间，则默认全天生效
+
+		return false
+	}
+
 	enableStime := strings.Fields(rule.EnableStime)
 	enableEtime := strings.Fields(rule.EnableEtime)
 	enableDaysOfWeek := strings.Split(rule.EnableDaysOfWeek, ";")

alert/pipeline/pipeline.go

@@ -1,12 +1,12 @@
 package pipeline
 
 import (
-	"github.com/ccfos/nightingale/v6/models"
-	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/aisummary"
+	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
+	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/eventdrop"
+	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/eventupdate"
+	_ "github.com/ccfos/nightingale/v6/alert/pipeline/processor/relabel"
 )
 
-func Pipeline(ctx *ctx.Context, event *models.AlertCurEvent, processors []Processor) {
-	for _, processor := range processors {
-		processor.Process(ctx, event)
-	}
+func Init() {
 }

alert/pipeline/processor/aisummary/ai_summary.go

@@ -0,0 +1,198 @@
+package aisummary
+
+import (
+	"bytes"
+	"crypto/tls"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strings"
+	"text/template"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	"github.com/ccfos/nightingale/v6/pkg/tplx"
+)
+
+const (
+	HTTP_STATUS_SUCCESS_MAX = 299
+)
+
+// AISummaryConfig 配置结构体
+type AISummaryConfig struct {
+	callback.HTTPConfig
+	ModelName      string                 `json:"model_name"`
+	APIKey         string                 `json:"api_key"`
+	PromptTemplate string                 `json:"prompt_template"`
+	CustomParams   map[string]interface{} `json:"custom_params"`
+}
+
+type Message struct {
+	Role    string `json:"role"`
+	Content string `json:"content"`
+}
+
+type ChatCompletionResponse struct {
+	Choices []struct {
+		Message struct {
+			Content string `json:"content"`
+		} `json:"message"`
+	} `json:"choices"`
+}
+
+func init() {
+	models.RegisterProcessor("ai_summary", &AISummaryConfig{})
+}
+
+func (c *AISummaryConfig) Init(settings interface{}) (models.Processor, error) {
+	result, err := common.InitProcessor[*AISummaryConfig](settings)
+	return result, err
+}
+
+func (c *AISummaryConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
+	if c.Client == nil {
+		if err := c.initHTTPClient(); err != nil {
+			return event, "", fmt.Errorf("failed to initialize HTTP client: %v processor: %v", err, c)
+		}
+	}
+
+	// 准备告警事件信息
+	eventInfo, err := c.prepareEventInfo(event)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to prepare event info: %v processor: %v", err, c)
+	}
+
+	// 调用AI模型生成总结
+	summary, err := c.generateAISummary(eventInfo)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to generate AI summary: %v processor: %v", err, c)
+	}
+
+	// 将总结添加到annotations字段
+	if event.AnnotationsJSON == nil {
+		event.AnnotationsJSON = make(map[string]string)
+	}
+	event.AnnotationsJSON["ai_summary"] = summary
+
+	// 更新Annotations字段
+	b, err := json.Marshal(event.AnnotationsJSON)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to marshal annotations: %v processor: %v", err, c)
+	}
+	event.Annotations = string(b)
+
+	return event, "", nil
+}
+
+func (c *AISummaryConfig) initHTTPClient() error {
+	transport := &http.Transport{
+		TLSClientConfig: &tls.Config{InsecureSkipVerify: c.SkipSSLVerify},
+	}
+
+	if c.Proxy != "" {
+		proxyURL, err := url.Parse(c.Proxy)
+		if err != nil {
+			return fmt.Errorf("failed to parse proxy url: %v", err)
+		}
+		transport.Proxy = http.ProxyURL(proxyURL)
+	}
+
+	c.Client = &http.Client{
+		Timeout:   time.Duration(c.Timeout) * time.Millisecond,
+		Transport: transport,
+	}
+	return nil
+}
+
+func (c *AISummaryConfig) prepareEventInfo(event *models.AlertCurEvent) (string, error) {
+	var defs = []string{
+		"{{$event := .}}",
+	}
+
+	text := strings.Join(append(defs, c.PromptTemplate), "")
+	t, err := template.New("prompt").Funcs(template.FuncMap(tplx.TemplateFuncMap)).Parse(text)
+	if err != nil {
+		return "", fmt.Errorf("failed to parse prompt template: %v", err)
+	}
+
+	var body bytes.Buffer
+	err = t.Execute(&body, event)
+	if err != nil {
+		return "", fmt.Errorf("failed to execute prompt template: %v", err)
+	}
+
+	return body.String(), nil
+}
+
+func (c *AISummaryConfig) generateAISummary(eventInfo string) (string, error) {
+	// 构建基础请求参数
+	reqParams := map[string]interface{}{
+		"model": c.ModelName,
+		"messages": []Message{
+			{
+				Role:    "user",
+				Content: eventInfo,
+			},
+		},
+	}
+
+	// 合并自定义参数
+	for k, v := range c.CustomParams {
+		reqParams[k] = v
+	}
+
+	// 序列化请求体
+	jsonData, err := json.Marshal(reqParams)
+	if err != nil {
+		return "", fmt.Errorf("failed to marshal request body: %v", err)
+	}
+
+	// 创建HTTP请求
+	req, err := http.NewRequest("POST", c.URL, bytes.NewBuffer(jsonData))
+	if err != nil {
+		return "", fmt.Errorf("failed to create request: %v", err)
+	}
+
+	// 设置请求头
+	req.Header.Set("Authorization", "Bearer "+c.APIKey)
+	req.Header.Set("Content-Type", "application/json")
+	for k, v := range c.Headers {
+		req.Header.Set(k, v)
+	}
+
+	// 发送请求
+	resp, err := c.Client.Do(req)
+	if err != nil {
+		return "", fmt.Errorf("failed to send request: %v", err)
+	}
+	defer resp.Body.Close()
+
+	// 检查响应状态码
+	if resp.StatusCode > HTTP_STATUS_SUCCESS_MAX {
+		body, _ := io.ReadAll(resp.Body)
+		return "", fmt.Errorf("unexpected status code: %d, body: %s", resp.StatusCode, string(body))
+	}
+
+	// 读取响应
+	body, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return "", fmt.Errorf("failed to read response body: %v", err)
+	}
+
+	// 解析响应
+	var chatResp ChatCompletionResponse
+	if err := json.Unmarshal(body, &chatResp); err != nil {
+		return "", fmt.Errorf("failed to unmarshal response: %v", err)
+	}
+
+	if len(chatResp.Choices) == 0 {
+		return "", fmt.Errorf("no response from AI model")
+	}
+
+	return chatResp.Choices[0].Message.Content, nil
+}

alert/pipeline/processor/aisummary/ai_summary_test.go

@@ -0,0 +1,69 @@
+package aisummary
+
+import (
+	"testing"
+
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestAISummaryConfig_Process(t *testing.T) {
+	// 创建测试配置
+	config := &AISummaryConfig{
+		HTTPConfig: callback.HTTPConfig{
+			URL:           "https://generativelanguage.googleapis.com/v1beta/openai/chat/completions",
+			Timeout:       30000,
+			SkipSSLVerify: true,
+			Headers: map[string]string{
+				"Content-Type": "application/json",
+			},
+		},
+		ModelName:      "gemini-2.0-flash",
+		APIKey:         "*",
+		PromptTemplate: "告警规则：{{$event.RuleName}}\n严重程度：{{$event.Severity}}",
+		CustomParams: map[string]interface{}{
+			"temperature": 0.7,
+			"max_tokens":  2000,
+			"top_p":       0.9,
+		},
+	}
+
+	// 创建测试事件
+	event := &models.AlertCurEvent{
+		RuleName: "Test Rule",
+		Severity: 1,
+		TagsMap: map[string]string{
+			"host": "test-host",
+		},
+		AnnotationsJSON: map[string]string{
+			"description": "Test alert",
+		},
+	}
+
+	// 测试模板处理
+	eventInfo, err := config.prepareEventInfo(event)
+	assert.NoError(t, err)
+	assert.Contains(t, eventInfo, "Test Rule")
+	assert.Contains(t, eventInfo, "1")
+
+	// 测试配置初始化
+	processor, err := config.Init(config)
+	assert.NoError(t, err)
+	assert.NotNil(t, processor)
+
+	// 测试处理函数
+	result, _, err := processor.Process(&ctx.Context{}, event)
+	assert.NoError(t, err)
+	assert.NotNil(t, result)
+	assert.NotEmpty(t, result.AnnotationsJSON["ai_summary"])
+
+	// 展示处理结果
+	t.Log("\n=== 处理结果 ===")
+	t.Logf("告警规则: %s", result.RuleName)
+	t.Logf("严重程度: %d", result.Severity)
+	t.Logf("标签: %v", result.TagsMap)
+	t.Logf("原始注释: %v", result.AnnotationsJSON["description"])
+	t.Logf("AI总结: %s", result.AnnotationsJSON["ai_summary"])
+}

alert/pipeline/processor/callback/callback.go

@@ -0,0 +1,103 @@
+package callback
+
+import (
+	"crypto/tls"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strings"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	"github.com/toolkits/pkg/logger"
+)
+
+type HTTPConfig struct {
+	URL           string            `json:"url"`
+	Method        string            `json:"method,omitempty"`
+	Body          string            `json:"body,omitempty"`
+	Headers       map[string]string `json:"header"`
+	AuthUsername  string            `json:"auth_username"`
+	AuthPassword  string            `json:"auth_password"`
+	Timeout       int               `json:"timeout"` // 单位:ms
+	SkipSSLVerify bool              `json:"skip_ssl_verify"`
+	Proxy         string            `json:"proxy"`
+	Client        *http.Client      `json:"-"`
+}
+
+// RelabelConfig
+type CallbackConfig struct {
+	HTTPConfig
+}
+
+func init() {
+	models.RegisterProcessor("callback", &CallbackConfig{})
+}
+
+func (c *CallbackConfig) Init(settings interface{}) (models.Processor, error) {
+	result, err := common.InitProcessor[*CallbackConfig](settings)
+	return result, err
+}
+
+func (c *CallbackConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
+	if c.Client == nil {
+		transport := &http.Transport{
+			TLSClientConfig: &tls.Config{InsecureSkipVerify: c.SkipSSLVerify},
+		}
+
+		if c.Proxy != "" {
+			proxyURL, err := url.Parse(c.Proxy)
+			if err != nil {
+				return event, "", fmt.Errorf("failed to parse proxy url: %v processor: %v", err, c)
+			} else {
+				transport.Proxy = http.ProxyURL(proxyURL)
+			}
+		}
+
+		c.Client = &http.Client{
+			Timeout:   time.Duration(c.Timeout) * time.Millisecond,
+			Transport: transport,
+		}
+	}
+
+	headers := make(map[string]string)
+	headers["Content-Type"] = "application/json"
+	for k, v := range c.Headers {
+		headers[k] = v
+	}
+
+	body, err := json.Marshal(event)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to marshal event: %v processor: %v", err, c)
+	}
+
+	req, err := http.NewRequest("POST", c.URL, strings.NewReader(string(body)))
+	if err != nil {
+		return event, "", fmt.Errorf("failed to create request: %v processor: %v", err, c)
+	}
+
+	for k, v := range headers {
+		req.Header.Set(k, v)
+	}
+
+	if c.AuthUsername != "" && c.AuthPassword != "" {
+		req.SetBasicAuth(c.AuthUsername, c.AuthPassword)
+	}
+
+	resp, err := c.Client.Do(req)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to send request: %v processor: %v", err, c)
+	}
+
+	b, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to read response body: %v processor: %v", err, c)
+	}
+
+	logger.Debugf("callback processor response body: %s", string(b))
+	return event, "callback success", nil
+}

alert/pipeline/processor/common/common.go

@@ -0,0 +1,24 @@
+package common
+
+import (
+	"encoding/json"
+)
+
+// InitProcessor 是一个通用的初始化处理器的方法
+// 使用泛型简化处理器初始化逻辑
+// T 必须是 models.Processor 接口的实现
+func InitProcessor[T any](settings interface{}) (T, error) {
+	var zero T
+	b, err := json.Marshal(settings)
+	if err != nil {
+		return zero, err
+	}
+
+	var result T
+	err = json.Unmarshal(b, &result)
+	if err != nil {
+		return zero, err
+	}
+
+	return result, nil
+}

alert/pipeline/processor/eventdrop/event_drop.go

@@ -0,0 +1,60 @@
+package eventdrop
+
+import (
+	"bytes"
+	"fmt"
+	"strings"
+	texttemplate "text/template"
+
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	"github.com/ccfos/nightingale/v6/pkg/tplx"
+	"github.com/toolkits/pkg/logger"
+)
+
+type EventDropConfig struct {
+	Content string `json:"content"`
+}
+
+func init() {
+	models.RegisterProcessor("event_drop", &EventDropConfig{})
+}
+
+func (c *EventDropConfig) Init(settings interface{}) (models.Processor, error) {
+	result, err := common.InitProcessor[*EventDropConfig](settings)
+	return result, err
+}
+
+func (c *EventDropConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
+	// 使用背景是可以根据此处理器，实现对事件进行更加灵活的过滤的逻辑
+	// 在标签过滤和属性过滤都不满足需求时可以使用
+	// 如果模板执行结果为 true，则删除该事件
+
+	var defs = []string{
+		"{{ $event := . }}",
+		"{{ $labels := .TagsMap }}",
+		"{{ $value := .TriggerValue }}",
+	}
+
+	text := strings.Join(append(defs, c.Content), "")
+
+	tpl, err := texttemplate.New("eventdrop").Funcs(tplx.TemplateFuncMap).Parse(text)
+	if err != nil {
+		return event, "", fmt.Errorf("processor failed to parse template: %v processor: %v", err, c)
+	}
+
+	var body bytes.Buffer
+	if err = tpl.Execute(&body, event); err != nil {
+		return event, "", fmt.Errorf("processor failed to execute template: %v processor: %v", err, c)
+	}
+
+	result := strings.TrimSpace(body.String())
+	logger.Infof("processor eventdrop result: %v", result)
+	if result == "true" {
+		logger.Infof("processor eventdrop drop event: %v", event)
+		return event, "drop event success", nil
+	}
+
+	return event, "drop event failed", nil
+}

alert/pipeline/processor/eventupdate/event_update.go

@@ -0,0 +1,96 @@
+package eventupdate
+
+import (
+	"crypto/tls"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strings"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/callback"
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	"github.com/toolkits/pkg/logger"
+)
+
+// RelabelConfig
+type EventUpdateConfig struct {
+	callback.HTTPConfig
+}
+
+func init() {
+	models.RegisterProcessor("event_update", &EventUpdateConfig{})
+}
+
+func (c *EventUpdateConfig) Init(settings interface{}) (models.Processor, error) {
+	result, err := common.InitProcessor[*EventUpdateConfig](settings)
+	return result, err
+}
+
+func (c *EventUpdateConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
+	if c.Client == nil {
+		transport := &http.Transport{
+			TLSClientConfig: &tls.Config{InsecureSkipVerify: c.SkipSSLVerify},
+		}
+
+		if c.Proxy != "" {
+			proxyURL, err := url.Parse(c.Proxy)
+			if err != nil {
+				return event, "", fmt.Errorf("failed to parse proxy url: %v processor: %v", err, c)
+			} else {
+				transport.Proxy = http.ProxyURL(proxyURL)
+			}
+		}
+
+		c.Client = &http.Client{
+			Timeout:   time.Duration(c.Timeout) * time.Millisecond,
+			Transport: transport,
+		}
+	}
+
+	headers := make(map[string]string)
+	headers["Content-Type"] = "application/json"
+	for k, v := range c.Headers {
+		headers[k] = v
+	}
+
+	body, err := json.Marshal(event)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to marshal event: %v processor: %v", err, c)
+	}
+
+	req, err := http.NewRequest("POST", c.URL, strings.NewReader(string(body)))
+	if err != nil {
+		return event, "", fmt.Errorf("failed to create request: %v processor: %v", err, c)
+	}
+
+	for k, v := range headers {
+		req.Header.Set(k, v)
+	}
+
+	if c.AuthUsername != "" && c.AuthPassword != "" {
+		req.SetBasicAuth(c.AuthUsername, c.AuthPassword)
+	}
+
+	resp, err := c.Client.Do(req)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to send request: %v processor: %v", err, c)
+	}
+
+	b, err := io.ReadAll(resp.Body)
+	if err != nil {
+		return nil, "", fmt.Errorf("failed to read response body: %v processor: %v", err, c)
+	}
+	logger.Debugf("event update processor response body: %s", string(b))
+
+	err = json.Unmarshal(b, &event)
+	if err != nil {
+		return event, "", fmt.Errorf("failed to unmarshal response body: %v processor: %v", err, c)
+	}
+
+	return event, "", nil
+}

alert/pipeline/processor/relabel/relabel.go

@@ -1,39 +1,70 @@
 package relabel
 
 import (
-	"encoding/json"
 	"fmt"
+	"regexp"
 	"strings"
 
-	"github.com/ccfos/nightingale/v6/alert/pipeline"
+	"github.com/ccfos/nightingale/v6/alert/pipeline/processor/common"
 	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/ctx"
 	"github.com/ccfos/nightingale/v6/pushgw/pconf"
 	"github.com/ccfos/nightingale/v6/pushgw/writer"
+
+	"github.com/prometheus/common/model"
 	"github.com/prometheus/prometheus/prompb"
 )
 
+const (
+	REPLACE_DOT = "___"
+)
+
 // RelabelConfig
 type RelabelConfig struct {
-	pconf.RelabelConfig
+	SourceLabels  []string `json:"source_labels"`
+	Separator     string   `json:"separator"`
+	Regex         string   `json:"regex"`
+	RegexCompiled *regexp.Regexp
+	If            string `json:"if"`
+	IfRegex       *regexp.Regexp
+	Modulus       uint64 `json:"modulus"`
+	TargetLabel   string `json:"target_label"`
+	Replacement   string `json:"replacement"`
+	Action        string `json:"action"`
 }
 
 func init() {
-	pipeline.RegisterProcessor("relabel", &RelabelConfig{})
+	models.RegisterProcessor("relabel", &RelabelConfig{})
 }
 
-func (r *RelabelConfig) Init(settings interface{}) (pipeline.Processor, error) {
-	b, err := json.Marshal(settings)
-	if err != nil {
-		return nil, err
+func (r *RelabelConfig) Init(settings interface{}) (models.Processor, error) {
+	result, err := common.InitProcessor[*RelabelConfig](settings)
+	return result, err
+}
+
+func (r *RelabelConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) (*models.AlertCurEvent, string, error) {
+	sourceLabels := make([]model.LabelName, len(r.SourceLabels))
+	for i := range r.SourceLabels {
+		sourceLabels[i] = model.LabelName(strings.ReplaceAll(r.SourceLabels[i], ".", REPLACE_DOT))
 	}
 
-	err = json.Unmarshal(b, &r.RelabelConfig)
-	return r, err
-}
+	relabelConfigs := []*pconf.RelabelConfig{
+		{
+			SourceLabels:  sourceLabels,
+			Separator:     r.Separator,
+			Regex:         r.Regex,
+			RegexCompiled: r.RegexCompiled,
+			If:            r.If,
+			IfRegex:       r.IfRegex,
+			Modulus:       r.Modulus,
+			TargetLabel:   r.TargetLabel,
+			Replacement:   r.Replacement,
+			Action:        r.Action,
+		},
+	}
 
-func (r *RelabelConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent) {
-	EventRelabel(event, []*pconf.RelabelConfig{&r.RelabelConfig})
+	EventRelabel(event, relabelConfigs)
+	return event, "", nil
 }
 
 func EventRelabel(event *models.AlertCurEvent, relabelConfigs []*pconf.RelabelConfig) {
@@ -41,7 +72,12 @@ func EventRelabel(event *models.AlertCurEvent, relabelConfigs []*pconf.RelabelCo
 	event.OriginalTagsJSON = make([]string, len(event.TagsJSON))
 	for i, tag := range event.TagsJSON {
 		label := strings.SplitN(tag, "=", 2)
+		if len(label) != 2 {
+			continue
+		}
 		event.OriginalTagsJSON[i] = tag
+
+		label[0] = strings.ReplaceAll(string(label[0]), ".", REPLACE_DOT)
 		labels[i] = prompb.Label{Name: label[0], Value: label[1]}
 	}
 
@@ -63,6 +99,7 @@ func EventRelabel(event *models.AlertCurEvent, relabelConfigs []*pconf.RelabelCo
 	event.TagsJSON = make([]string, len(gotLabels))
 	event.TagsMap = make(map[string]string, len(gotLabels))
 	for i, label := range gotLabels {
+		label.Name = strings.ReplaceAll(string(label.Name), REPLACE_DOT, ".")
 		event.TagsJSON[i] = fmt.Sprintf("%s=%s", label.Name, label.Value)
 		event.TagsMap[label.Name] = label.Value
 	}

alert/process/process.go

@@ -60,11 +60,9 @@ type Processor struct {
 	pendingsUseByRecover *AlertCurEventMap
 	inhibit              bool
 
-	tagsMap    map[string]string
-	tagsArr    []string
-	target     string
-	targetNote string
-	groupName  string
+	tagsMap   map[string]string
+	tagsArr   []string
+	groupName string
 
 	alertRuleCache          *memsto.AlertRuleCacheType
 	TargetCache             *memsto.TargetCacheType
@@ -153,7 +151,7 @@ func (p *Processor) Handle(anomalyPoints []models.AnomalyPoint, from string, inh
 	eventsMap := make(map[string][]*models.AlertCurEvent)
 	for _, anomalyPoint := range anomalyPoints {
 		event := p.BuildEvent(anomalyPoint, from, now, ruleHash)
-		event.NotifyRuleIDs = cachedRule.NotifyRuleIds
+		event.NotifyRuleIds = cachedRule.NotifyRuleIds
 		// 如果 event 被 mute 了,本质也是 fire 的状态,这里无论如何都添加到 alertingKeys 中,防止 fire 的事件自动恢复了
 		hash := event.Hash
 		alertingKeys[hash] = struct{}{}
@@ -195,7 +193,7 @@ func (p *Processor) Handle(anomalyPoints []models.AnomalyPoint, from string, inh
 
 func (p *Processor) BuildEvent(anomalyPoint models.AnomalyPoint, from string, now int64, ruleHash string) *models.AlertCurEvent {
 	p.fillTags(anomalyPoint)
-	p.mayHandleIdent()
+
 	hash := Hash(p.rule.Id, p.datasourceId, anomalyPoint)
 	ds := p.datasourceCache.GetById(p.datasourceId)
 	var dsName string
@@ -215,8 +213,6 @@ func (p *Processor) BuildEvent(anomalyPoint models.AnomalyPoint, from string, no
 	event.DatasourceId = p.datasourceId
 	event.Cluster = dsName
 	event.Hash = hash
-	event.TargetIdent = p.target
-	event.TargetNote = p.targetNote
 	event.TriggerValue = anomalyPoint.ReadableValue()
 	event.TriggerValues = anomalyPoint.Values
 	event.TriggerValuesJson = models.EventTriggerValues{ValuesWithUnit: anomalyPoint.ValuesUnit}
@@ -248,15 +244,6 @@ func (p *Processor) BuildEvent(anomalyPoint models.AnomalyPoint, from string, no
 		logger.Warningf("unmarshal annotations json failed: %v, rule: %d", err, p.rule.Id)
 	}
 
-	if p.target != "" {
-		if pt, exist := p.TargetCache.Get(p.target); exist {
-			pt.GroupNames = p.BusiGroupCache.GetNamesByBusiGroupIds(pt.GroupIds)
-			event.Target = pt
-		} else {
-			logger.Infof("Target[ident: %s] doesn't exist in cache.", p.target)
-		}
-	}
-
 	if event.TriggerValues != "" && strings.Count(event.TriggerValues, "$") > 1 {
 		// TriggerValues 有多个变量，将多个变量都放到 TriggerValue 中
 		event.TriggerValue = event.TriggerValues
@@ -270,6 +257,19 @@ func (p *Processor) BuildEvent(anomalyPoint models.AnomalyPoint, from string, no
 
 	// 生成事件之后，立马进程 relabel 处理
 	Relabel(p.rule, event)
+
+	// 放到 Relabel(p.rule, event) 下面，为了处理 relabel 之后，标签里才出现 ident 的情况
+	p.mayHandleIdent(event)
+
+	if event.TargetIdent != "" {
+		if pt, exist := p.TargetCache.Get(event.TargetIdent); exist {
+			pt.GroupNames = p.BusiGroupCache.GetNamesByBusiGroupIds(pt.GroupIds)
+			event.Target = pt
+		} else {
+			logger.Infof("fill event target error, ident: %s doesn't exist in cache.", event.TargetIdent)
+		}
+	}
+
 	return event
 }
 
@@ -280,7 +280,7 @@ func Relabel(rule *models.AlertRule, event *models.AlertCurEvent) {
 
 	// need to keep the original label
 	event.OriginalTags = event.Tags
-	event.OriginalTagsJSON = make([]string, len(event.TagsJSON))
+	event.OriginalTagsJSON = event.TagsJSON
 
 	if len(rule.EventRelabelConfig) == 0 {
 		return
@@ -537,7 +537,7 @@ func (p *Processor) RecoverAlertCurEventFromDb() {
 		if alertRule == nil {
 			continue
 		}
-		event.NotifyRuleIDs = alertRule.NotifyRuleIds
+		event.NotifyRuleIds = alertRule.NotifyRuleIds
 
 		if event.Cate == models.HOST {
 			target, exists := p.TargetCache.Get(event.TargetIdent)
@@ -611,19 +611,19 @@ func (p *Processor) fillTags(anomalyPoint models.AnomalyPoint) {
 	p.tagsArr = labelMapToArr(tagsMap)
 }
 
-func (p *Processor) mayHandleIdent() {
+func (p *Processor) mayHandleIdent(event *models.AlertCurEvent) {
 	// handle ident
-	if ident, has := p.tagsMap["ident"]; has {
+	if ident, has := event.TagsMap["ident"]; has {
 		if target, exists := p.TargetCache.Get(ident); exists {
-			p.target = target.Ident
-			p.targetNote = target.Note
+			event.TargetIdent = target.Ident
+			event.TargetNote = target.Note
 		} else {
-			p.target = ident
-			p.targetNote = ""
+			event.TargetIdent = ident
+			event.TargetNote = ""
 		}
 	} else {
-		p.target = ""
-		p.targetNote = ""
+		event.TargetIdent = ""
+		event.TargetNote = ""
 	}
 }
 

alert/sender/ibex.go

@@ -30,12 +30,14 @@ type IbexCallBacker struct {
 
 func (c *IbexCallBacker) CallBack(ctx CallBackContext) {
 	if len(ctx.CallBackURL) == 0 || len(ctx.Events) == 0 {
+		logger.Warningf("event_callback_ibex: url or events is empty, url: %s, events: %+v", ctx.CallBackURL, ctx.Events)
 		return
 	}
 
 	event := ctx.Events[0]
 
 	if event.IsRecovered {
+		logger.Infof("event_callback_ibex: event is recovered, event: %+v", event)
 		return
 	}
 
@@ -43,8 +45,9 @@ func (c *IbexCallBacker) CallBack(ctx CallBackContext) {
 }
 
 func (c *IbexCallBacker) handleIbex(ctx *ctx.Context, url string, event *models.AlertCurEvent) {
+	logger.Infof("event_callback_ibex: url: %s, event: %+v", url, event)
 	if imodels.DB() == nil && ctx.IsCenter {
-		logger.Warning("event_callback_ibex: db is nil")
+		logger.Warningf("event_callback_ibex: db is nil, event: %+v", event)
 		return
 	}
 
@@ -63,17 +66,23 @@ func (c *IbexCallBacker) handleIbex(ctx *ctx.Context, url string, event *models.
 
 	id, err := strconv.ParseInt(idstr, 10, 64)
 	if err != nil {
-		logger.Errorf("event_callback_ibex: failed to parse url: %s", url)
+		logger.Errorf("event_callback_ibex: failed to parse url: %s event: %+v", url, event)
 		return
 	}
 
 	if host == "" {
 		// 用户在callback url中没有传入host，就从event中解析
 		host = event.TargetIdent
+
+		if host == "" {
+			if ident, has := event.TagsMap["ident"]; has {
+				host = ident
+			}
+		}
 	}
 
 	if host == "" {
-		logger.Error("event_callback_ibex: failed to get host")
+		logger.Errorf("event_callback_ibex: failed to get host, id: %d, event: %+v", id, event)
 		return
 	}
 
@@ -83,21 +92,23 @@ func (c *IbexCallBacker) handleIbex(ctx *ctx.Context, url string, event *models.
 func CallIbex(ctx *ctx.Context, id int64, host string,
 	taskTplCache *memsto.TaskTplCache, targetCache *memsto.TargetCacheType,
 	userCache *memsto.UserCacheType, event *models.AlertCurEvent) {
+	logger.Infof("event_callback_ibex: id: %d, host: %s, event: %+v", id, host, event)
+
 	tpl := taskTplCache.Get(id)
 	if tpl == nil {
-		logger.Errorf("event_callback_ibex: no such tpl(%d)", id)
+		logger.Errorf("event_callback_ibex: no such tpl(%d), event: %+v", id, event)
 		return
 	}
 	// check perm
 	// tpl.GroupId - host - account 三元组校验权限
 	can, err := canDoIbex(tpl.UpdateBy, tpl, host, targetCache, userCache)
 	if err != nil {
-		logger.Errorf("event_callback_ibex: check perm fail: %v", err)
+		logger.Errorf("event_callback_ibex: check perm fail: %v, event: %+v", err, event)
 		return
 	}
 
 	if !can {
-		logger.Errorf("event_callback_ibex: user(%s) no permission", tpl.UpdateBy)
+		logger.Errorf("event_callback_ibex: user(%s) no permission, event: %+v", tpl.UpdateBy, event)
 		return
 	}
 
@@ -122,7 +133,7 @@ func CallIbex(ctx *ctx.Context, id int64, host string,
 
 	tags, err := json.Marshal(tagsMap)
 	if err != nil {
-		logger.Errorf("event_callback_ibex: failed to marshal tags to json: %v", tagsMap)
+		logger.Errorf("event_callback_ibex: failed to marshal tags to json: %v, event: %+v", tagsMap, event)
 		return
 	}
 
@@ -145,7 +156,7 @@ func CallIbex(ctx *ctx.Context, id int64, host string,
 
 	id, err = TaskAdd(in, tpl.UpdateBy, ctx.IsCenter)
 	if err != nil {
-		logger.Errorf("event_callback_ibex: call ibex fail: %v", err)
+		logger.Errorf("event_callback_ibex: call ibex fail: %v, event: %+v", err, event)
 		return
 	}
 
@@ -167,7 +178,7 @@ func CallIbex(ctx *ctx.Context, id int64, host string,
 	}
 
 	if err = record.Add(ctx); err != nil {
-		logger.Errorf("event_callback_ibex: persist task_record fail: %v", err)
+		logger.Errorf("event_callback_ibex: persist task_record fail: %v, event: %+v", err, event)
 	}
 }
 
@@ -187,7 +198,7 @@ func canDoIbex(username string, tpl *models.TaskTpl, host string, targetCache *m
 
 func TaskAdd(f models.TaskForm, authUser string, isCenter bool) (int64, error) {
 	if storage.Cache == nil {
-		logger.Warning("event_callback_ibex: redis cache is nil")
+		logger.Warningf("event_callback_ibex: redis cache is nil, task: %+v", f)
 		return 0, fmt.Errorf("redis cache is nil")
 	}
 

center/cconf/ops.go

@@ -256,6 +256,12 @@ ops:
   ops:
     - name: /users
       cname: User - View
+    - name: /users/add
+      cname: User - Add
+    - name: /users/put
+      cname: User - Modify
+    - name: /users/del
+      cname: User - Delete
     - name: /user-groups
       cname: Team - View
     - name: /user-groups/add
@@ -274,6 +280,12 @@ ops:
       cname: Business Group - Delete
     - name: /roles
       cname: Role - View
+    - name: /roles/add
+      cname: Role - Add
+    - name: /roles/put
+      cname: Role - Modify
+    - name: /roles/del
+      cname: Role - Delete
 
 - name: System Settings
   cname: System Settings

center/cconf/plugin.go

@@ -25,4 +25,22 @@ var Plugins = []Plugin{
 		Type:     "tdengine",
 		TypeName: "TDengine",
 	},
+	{
+		Id:       5,
+		Category: "logging",
+		Type:     "ck",
+		TypeName: "ClickHouse",
+	},
+	{
+		Id:       6,
+		Category: "timeseries",
+		Type:     "mysql",
+		TypeName: "MySQL",
+	},
+	{
+		Id:       7,
+		Category: "timeseries",
+		Type:     "pgsql",
+		TypeName: "PostgreSQL",
+	},
 }

center/router/router.go

@@ -264,11 +264,11 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.DELETE("/self/token/:id", rt.auth(), rt.user(), rt.deleteToken)
 
 		pages.GET("/users", rt.auth(), rt.user(), rt.perm("/users"), rt.userGets)
-		pages.POST("/users", rt.auth(), rt.admin(), rt.userAddPost)
+		pages.POST("/users", rt.auth(), rt.user(), rt.perm("/users/add"), rt.userAddPost)
 		pages.GET("/user/:id/profile", rt.auth(), rt.userProfileGet)
-		pages.PUT("/user/:id/profile", rt.auth(), rt.admin(), rt.userProfilePut)
-		pages.PUT("/user/:id/password", rt.auth(), rt.admin(), rt.userPasswordPut)
-		pages.DELETE("/user/:id", rt.auth(), rt.admin(), rt.userDel)
+		pages.PUT("/user/:id/profile", rt.auth(), rt.user(), rt.perm("/users/put"), rt.userProfilePut)
+		pages.PUT("/user/:id/password", rt.auth(), rt.user(), rt.perm("/users/put"), rt.userPasswordPut)
+		pages.DELETE("/user/:id", rt.auth(), rt.user(), rt.perm("/users/del"), rt.userDel)
 
 		pages.GET("/metric-views", rt.auth(), rt.metricViewGets)
 		pages.DELETE("/metric-views", rt.auth(), rt.user(), rt.metricViewDel)
@@ -398,24 +398,18 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.PUT("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/put"), rt.bgrw(), rt.alertSubscribePut)
 		pages.DELETE("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/del"), rt.bgrw(), rt.alertSubscribeDel)
 
-		if rt.Center.AnonymousAccess.AlertDetail {
-			pages.GET("/alert-cur-event/:eid", rt.alertCurEventGet)
-			pages.GET("/alert-his-event/:eid", rt.alertHisEventGet)
-			pages.GET("/event-notify-records/:eid", rt.notificationRecordList)
-		} else {
-			pages.GET("/alert-cur-event/:eid", rt.auth(), rt.user(), rt.alertCurEventGet)
-			pages.GET("/alert-his-event/:eid", rt.auth(), rt.user(), rt.alertHisEventGet)
-			pages.GET("/event-notify-records/:eid", rt.auth(), rt.user(), rt.notificationRecordList)
-		}
+		pages.GET("/alert-cur-event/:eid", rt.alertCurEventGet)
+		pages.GET("/alert-his-event/:eid", rt.alertHisEventGet)
+		pages.GET("/event-notify-records/:eid", rt.notificationRecordList)
 
 		// card logic
 		pages.GET("/alert-cur-events/list", rt.auth(), rt.user(), rt.alertCurEventsList)
 		pages.GET("/alert-cur-events/card", rt.auth(), rt.user(), rt.alertCurEventsCard)
 		pages.POST("/alert-cur-events/card/details", rt.auth(), rt.alertCurEventsCardDetails)
 		pages.GET("/alert-his-events/list", rt.auth(), rt.user(), rt.alertHisEventsList)
+		pages.DELETE("/alert-his-events", rt.auth(), rt.admin(), rt.alertHisEventsDelete)
 		pages.DELETE("/alert-cur-events", rt.auth(), rt.user(), rt.perm("/alert-cur-events/del"), rt.alertCurEventDel)
 		pages.GET("/alert-cur-events/stats", rt.auth(), rt.alertCurEventsStatistics)
-		pages.GET("/alert-cur-events-datasources", rt.auth(), rt.user(), rt.alertDataSourcesList)
 
 		pages.GET("/alert-aggr-views", rt.auth(), rt.alertAggrViewGets)
 		pages.DELETE("/alert-aggr-views", rt.auth(), rt.user(), rt.alertAggrViewDel)
@@ -446,12 +440,12 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.DELETE("/datasource/", rt.auth(), rt.admin(), rt.datasourceDel)
 
 		pages.GET("/roles", rt.auth(), rt.user(), rt.perm("/roles"), rt.roleGets)
-		pages.POST("/roles", rt.auth(), rt.admin(), rt.roleAdd)
-		pages.PUT("/roles", rt.auth(), rt.admin(), rt.rolePut)
-		pages.DELETE("/role/:id", rt.auth(), rt.admin(), rt.roleDel)
+		pages.POST("/roles", rt.auth(), rt.user(), rt.perm("/roles/add"), rt.roleAdd)
+		pages.PUT("/roles", rt.auth(), rt.user(), rt.perm("/roles/put"), rt.rolePut)
+		pages.DELETE("/role/:id", rt.auth(), rt.user(), rt.perm("/roles/del"), rt.roleDel)
 
 		pages.GET("/role/:id/ops", rt.auth(), rt.user(), rt.perm("/roles"), rt.operationOfRole)
-		pages.PUT("/role/:id/ops", rt.auth(), rt.admin(), rt.roleBindOperation)
+		pages.PUT("/role/:id/ops", rt.auth(), rt.user(), rt.perm("/roles/put"), rt.roleBindOperation)
 		pages.GET("/operation", rt.operations)
 
 		pages.GET("/notify-tpls", rt.auth(), rt.user(), rt.notifyTplGets)
@@ -505,6 +499,9 @@ func (rt *Router) Config(r *gin.Engine) {
 		pages.PUT("/config", rt.auth(), rt.admin(), rt.configPutByKey)
 		pages.GET("/site-info", rt.siteInfo)
 
+		// source token 相关路由
+		pages.POST("/source-token", rt.auth(), rt.user(), rt.sourceTokenAdd)
+
 		// for admin api
 		pages.GET("/user/busi-groups", rt.auth(), rt.admin(), rt.userBusiGroupsGets)
 

center/router/router_alert_cur_event.go

@@ -1,44 +1,20 @@
 package router
 
 import (
+	"fmt"
 	"net/http"
 	"sort"
 	"strings"
 	"time"
 
 	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+	"github.com/ccfos/nightingale/v6/pkg/strx"
 
 	"github.com/gin-gonic/gin"
 	"github.com/toolkits/pkg/ginx"
 )
 
-func parseAggrRules(rule string) []*models.AggrRule {
-	aggrRules := strings.Split(rule, "::") // e.g. field:group_name::field:severity::tagkey:ident
-
-	if len(aggrRules) == 0 {
-		ginx.Bomb(http.StatusBadRequest, "rule empty")
-	}
-
-	rules := make([]*models.AggrRule, len(aggrRules))
-	for i := 0; i < len(aggrRules); i++ {
-		pair := strings.Split(aggrRules[i], ":")
-		if len(pair) != 2 {
-			ginx.Bomb(http.StatusBadRequest, "rule invalid")
-		}
-
-		if !(pair[0] == "field" || pair[0] == "tagkey") {
-			ginx.Bomb(http.StatusBadRequest, "rule invalid")
-		}
-
-		rules[i] = &models.AggrRule{
-			Type:  pair[0],
-			Value: pair[1],
-		}
-	}
-
-	return rules
-}
-
 func getUserGroupIds(ctx *gin.Context, rt *Router, myGroups bool) ([]int64, error) {
 	if !myGroups {
 		return nil, nil
@@ -49,7 +25,7 @@ func getUserGroupIds(ctx *gin.Context, rt *Router, myGroups bool) ([]int64, erro
 
 func (rt *Router) alertCurEventsCard(c *gin.Context) {
 	stime, etime := getTimeRange(c)
-	severity := ginx.QueryInt(c, "severity", -1)
+	severity := strx.IdsInt64ForAPI(ginx.QueryStr(c, "severity", ""), ",")
 	query := ginx.QueryStr(c, "query", "")
 	myGroups := ginx.QueryBool(c, "my_groups", false) // 是否只看自己组，默认false
 
@@ -74,8 +50,6 @@ func (rt *Router) alertCurEventsCard(c *gin.Context) {
 
 	dsIds := queryDatasourceIds(c)
 
-	rules := parseAggrRules(alertView.Rule)
-
 	prod := ginx.QueryStr(c, "prods", "")
 	if prod == "" {
 		prod = ginx.QueryStr(c, "rule_prods", "")
@@ -91,17 +65,17 @@ func (rt *Router) alertCurEventsCard(c *gin.Context) {
 		cates = strings.Split(cate, ",")
 	}
 
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
+	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView, myGroups)
 	ginx.Dangerous(err)
 
 	// 最多获取50000个，获取太多也没啥意义
 	list, err := models.AlertCurEventsGet(rt.Ctx, prods, bgids, stime, etime, severity, dsIds,
-		cates, 0, query, 50000, 0, gids)
+		cates, 0, query, 50000, 0, []int64{})
 	ginx.Dangerous(err)
 
 	cardmap := make(map[string]*AlertCard)
 	for _, event := range list {
-		title, err := event.GenCardTitle(rules, alertView.Format)
+		title, err := event.GenCardTitle(alertView.Rule)
 		ginx.Dangerous(err)
 		if _, has := cardmap[title]; has {
 			cardmap[title].Total++
@@ -117,6 +91,10 @@ func (rt *Router) alertCurEventsCard(c *gin.Context) {
 				Severity: event.Severity,
 			}
 		}
+
+		if cardmap[title].Severity < 1 {
+			cardmap[title].Severity = 3
+		}
 	}
 
 	titles := make([]string, 0, len(cardmap))
@@ -173,13 +151,15 @@ func (rt *Router) alertCurEventsGetByRid(c *gin.Context) {
 // 列表方式，拉取活跃告警
 func (rt *Router) alertCurEventsList(c *gin.Context) {
 	stime, etime := getTimeRange(c)
-	severity := ginx.QueryInt(c, "severity", -1)
+	severity := strx.IdsInt64ForAPI(ginx.QueryStr(c, "severity", ""), ",")
 	query := ginx.QueryStr(c, "query", "")
 	limit := ginx.QueryInt(c, "limit", 20)
 	myGroups := ginx.QueryBool(c, "my_groups", false) // 是否只看自己组，默认false
 
 	dsIds := queryDatasourceIds(c)
 
+	eventIds := strx.IdsInt64ForAPI(ginx.QueryStr(c, "event_ids", ""), ",")
+
 	prod := ginx.QueryStr(c, "prods", "")
 	if prod == "" {
 		prod = ginx.QueryStr(c, "rule_prods", "")
@@ -198,25 +178,15 @@ func (rt *Router) alertCurEventsList(c *gin.Context) {
 
 	ruleId := ginx.QueryInt64(c, "rid", 0)
 
-	var gids []int64
-	var err error
-	if myGroups {
-		gids, err = getUserGroupIds(c, rt, myGroups)
-		ginx.Dangerous(err)
-		if len(gids) == 0 {
-			gids = append(gids, -1)
-		}
-	}
-
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
+	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView, myGroups)
 	ginx.Dangerous(err)
 
 	total, err := models.AlertCurEventTotal(rt.Ctx, prods, bgids, stime, etime, severity, dsIds,
-		cates, ruleId, query, gids)
+		cates, ruleId, query, eventIds)
 	ginx.Dangerous(err)
 
 	list, err := models.AlertCurEventsGet(rt.Ctx, prods, bgids, stime, etime, severity, dsIds,
-		cates, ruleId, query, limit, ginx.Offset(c, limit), gids)
+		cates, ruleId, query, limit, ginx.Offset(c, limit), eventIds)
 	ginx.Dangerous(err)
 
 	cache := make(map[int64]*models.UserGroup)
@@ -260,83 +230,70 @@ func (rt *Router) checkCurEventBusiGroupRWPermission(c *gin.Context, ids []int64
 	}
 }
 
-// 列表方式，拉取活跃告警
-func (rt *Router) alertDataSourcesList(c *gin.Context) {
-	stime, etime := getTimeRange(c)
-	severity := ginx.QueryInt(c, "severity", -1)
-	query := ginx.QueryStr(c, "query", "")
-	myGroups := ginx.QueryBool(c, "my_groups", false) // 是否只看自己组，默认false
-
-	prod := ginx.QueryStr(c, "prods", "")
-	if prod == "" {
-		prod = ginx.QueryStr(c, "rule_prods", "")
-	}
-
-	prods := []string{}
-	if prod != "" {
-		prods = strings.Split(prod, ",")
-	}
-
-	cate := ginx.QueryStr(c, "cate", "$all")
-	cates := []string{}
-	if cate != "$all" {
-		cates = strings.Split(cate, ",")
-	}
-
-	ruleId := ginx.QueryInt64(c, "rid", 0)
-	var gids []int64
-	var err error
-	if myGroups {
-		gids, err = getUserGroupIds(c, rt, myGroups)
-		ginx.Dangerous(err)
-		if len(gids) == 0 {
-			gids = append(gids, -1)
-		}
-	}
-
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
-	ginx.Dangerous(err)
-
-	list, err := models.AlertCurEventsGet(rt.Ctx, prods, bgids, stime, etime, severity, []int64{},
-		cates, ruleId, query, 50000, 0, gids)
-	ginx.Dangerous(err)
-
-	uniqueDsIds := make(map[int64]struct{})
-
-	for i := 0; i < len(list); i++ {
-		uniqueDsIds[list[i].DatasourceId] = struct{}{}
-	}
-
-	dsIds := make([]int64, 0, len(uniqueDsIds))
-	for id := range uniqueDsIds {
-		dsIds = append(dsIds, id)
-	}
-	dsList, err := models.GetDatasourceInfosByIds(rt.Ctx, dsIds)
-	ginx.Dangerous(err)
-
-	ginx.NewRender(c).Data(dsList, nil)
-}
-
 func (rt *Router) alertCurEventGet(c *gin.Context) {
 	eid := ginx.UrlParamInt64(c, "eid")
-	event, err := models.AlertCurEventGetById(rt.Ctx, eid)
-	ginx.Dangerous(err)
+	event, err := GetCurEventDetail(rt.Ctx, eid)
 
-	if event == nil {
-		ginx.Bomb(404, "No such active event")
-	}
-
-	if !rt.Center.AnonymousAccess.AlertDetail && rt.Center.EventHistoryGroupView {
+	hasPermission := HasPermission(rt.Ctx, c, "event", fmt.Sprintf("%d", eid), rt.Center.AnonymousAccess.AlertDetail)
+	if !hasPermission {
+		rt.auth()(c)
+		rt.user()(c)
 		rt.bgroCheck(c, event.GroupId)
 	}
 
-	ruleConfig, needReset := models.FillRuleConfigTplName(rt.Ctx, event.RuleConfig)
+	ginx.NewRender(c).Data(event, err)
+}
+
+func GetCurEventDetail(ctx *ctx.Context, eid int64) (*models.AlertCurEvent, error) {
+	event, err := models.AlertCurEventGetById(ctx, eid)
+	if err != nil {
+		return nil, err
+	}
+
+	if event == nil {
+		return nil, fmt.Errorf("no such active event")
+	}
+
+	ruleConfig, needReset := models.FillRuleConfigTplName(ctx, event.RuleConfig)
 	if needReset {
 		event.RuleConfigJson = ruleConfig
 	}
 
 	event.LastEvalTime = event.TriggerTime
-	ginx.NewRender(c).Data(event, nil)
+	event.NotifyVersion, err = GetEventNotifyVersion(ctx, event.RuleId, event.NotifyRuleIds)
+	ginx.Dangerous(err)
+
+	event.NotifyRules, err = GetEventNorifyRuleNames(ctx, event.NotifyRuleIds)
+	return event, err
+}
+
+func GetEventNorifyRuleNames(ctx *ctx.Context, notifyRuleIds []int64) ([]*models.EventNotifyRule, error) {
+	notifyRuleNames := make([]*models.EventNotifyRule, 0)
+	notifyRules, err := models.NotifyRulesGet(ctx, "id in ?", notifyRuleIds)
+	if err != nil {
+		return nil, err
+	}
+
+	for _, notifyRule := range notifyRules {
+		notifyRuleNames = append(notifyRuleNames, &models.EventNotifyRule{
+			Id:   notifyRule.ID,
+			Name: notifyRule.Name,
+		})
+	}
+	return notifyRuleNames, nil
+}
+
+func GetEventNotifyVersion(ctx *ctx.Context, ruleId int64, notifyRuleIds []int64) (int, error) {
+	if len(notifyRuleIds) != 0 {
+		// 如果存在 notify_rule_ids，则认为使用新的告警通知方式
+		return 1, nil
+	}
+
+	rule, err := models.AlertRuleGetById(ctx, ruleId)
+	if err != nil {
+		return 0, err
+	}
+	return rule.NotifyVersion, nil
 }
 
 func (rt *Router) alertCurEventsStatistics(c *gin.Context) {

center/router/router_alert_his_event.go

@@ -2,6 +2,7 @@ package router
 
 import (
 	"fmt"
+	"net/http"
 	"strings"
 	"time"
 
@@ -10,6 +11,7 @@ import (
 
 	"github.com/gin-gonic/gin"
 	"github.com/toolkits/pkg/ginx"
+	"github.com/toolkits/pkg/logger"
 	"golang.org/x/exp/slices"
 )
 
@@ -56,7 +58,7 @@ func (rt *Router) alertHisEventsList(c *gin.Context) {
 
 	ruleId := ginx.QueryInt64(c, "rid", 0)
 
-	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView)
+	bgids, err := GetBusinessGroupIds(c, rt.Ctx, rt.Center.EventHistoryGroupView, false)
 	ginx.Dangerous(err)
 
 	total, err := models.AlertHisEventTotal(rt.Ctx, prods, bgids, stime, etime, severity,
@@ -78,16 +80,56 @@ func (rt *Router) alertHisEventsList(c *gin.Context) {
 	}, nil)
 }
 
+type alertHisEventsDeleteForm struct {
+	Severities []int `json:"severities"`
+	Timestamp  int64 `json:"timestamp" binding:"required"`
+}
+
+func (rt *Router) alertHisEventsDelete(c *gin.Context) {
+	var f alertHisEventsDeleteForm
+	ginx.BindJSON(c, &f)
+	// 校验
+	if f.Timestamp == 0 {
+		ginx.Bomb(http.StatusBadRequest, "timestamp parameter is required")
+		return
+	}
+
+	user := c.MustGet("user").(*models.User)
+
+	// 启动后台清理任务
+	go func() {
+		limit := 100
+		for {
+			n, err := models.AlertHisEventBatchDelete(rt.Ctx, f.Timestamp, f.Severities, limit)
+			if err != nil {
+				logger.Errorf("Failed to delete alert history events: operator=%s, timestamp=%d, severities=%v, error=%v",
+					user.Username, f.Timestamp, f.Severities, err)
+				break
+			}
+			logger.Debugf("Successfully deleted alert history events: operator=%s, timestamp=%d, severities=%v, deleted=%d",
+				user.Username, f.Timestamp, f.Severities, n)
+			if n < int64(limit) {
+				break // 已经删完
+			}
+
+			time.Sleep(100 * time.Millisecond) // 防止锁表
+		}
+	}()
+	ginx.NewRender(c).Message("Alert history events deletion started")
+}
+
 func (rt *Router) alertHisEventGet(c *gin.Context) {
 	eid := ginx.UrlParamInt64(c, "eid")
 	event, err := models.AlertHisEventGetById(rt.Ctx, eid)
 	ginx.Dangerous(err)
-
 	if event == nil {
 		ginx.Bomb(404, "No such alert event")
 	}
 
-	if !rt.Center.AnonymousAccess.AlertDetail && rt.Center.EventHistoryGroupView {
+	hasPermission := HasPermission(rt.Ctx, c, "event", fmt.Sprintf("%d", eid), rt.Center.AnonymousAccess.AlertDetail)
+	if !hasPermission {
+		rt.auth()(c)
+		rt.user()(c)
 		rt.bgroCheck(c, event.GroupId)
 	}
 
@@ -96,46 +138,54 @@ func (rt *Router) alertHisEventGet(c *gin.Context) {
 		event.RuleConfigJson = ruleConfig
 	}
 
+	event.NotifyVersion, err = GetEventNotifyVersion(rt.Ctx, event.RuleId, event.NotifyRuleIds)
+	ginx.Dangerous(err)
+
+	event.NotifyRules, err = GetEventNorifyRuleNames(rt.Ctx, event.NotifyRuleIds)
 	ginx.NewRender(c).Data(event, err)
 }
 
-func GetBusinessGroupIds(c *gin.Context, ctx *ctx.Context, eventHistoryGroupView bool) ([]int64, error) {
+func GetBusinessGroupIds(c *gin.Context, ctx *ctx.Context, onlySelfGroupView bool, myGroups bool) ([]int64, error) {
 	bgid := ginx.QueryInt64(c, "bgid", 0)
 	var bgids []int64
 
-	if !eventHistoryGroupView || strings.HasPrefix(c.Request.URL.Path, "/v1") {
+	if strings.HasPrefix(c.Request.URL.Path, "/v1") {
+		// 如果请求路径以 /v1 开头，不查询用户信息
 		if bgid > 0 {
 			return []int64{bgid}, nil
 		}
+
 		return bgids, nil
 	}
 
 	user := c.MustGet("user").(*models.User)
-	if user.IsAdmin() {
+	if myGroups || (onlySelfGroupView && !user.IsAdmin()) {
+		// 1. 页面上勾选了我的业务组，需要查询用户所属的业务组
+		// 2. 如果 onlySelfGroupView 为 true，表示只允许查询用户所属的业务组
+		bussGroupIds, err := models.MyBusiGroupIds(ctx, user.Id)
+		if err != nil {
+			return nil, err
+		}
+
+		if len(bussGroupIds) == 0 {
+			// 如果没查到用户属于任何业务组，需要返回一个0，否则会导致查询到全部告警历史
+			return []int64{0}, nil
+		}
+
 		if bgid > 0 {
+			if !slices.Contains(bussGroupIds, bgid) && !user.IsAdmin() {
+				return nil, fmt.Errorf("business group ID not allowed")
+			}
+
 			return []int64{bgid}, nil
 		}
-		return bgids, nil
-	}
 
-	bussGroupIds, err := models.MyBusiGroupIds(ctx, user.Id)
-	if err != nil {
-		return nil, err
-	}
-
-	if len(bussGroupIds) == 0 {
-		// 如果没查到用户属于任何业务组，需要返回一个0，否则会导致查询到全部告警历史
-		return []int64{0}, nil
-	}
-
-	if bgid > 0 && !slices.Contains(bussGroupIds, bgid) {
-		return nil, fmt.Errorf("business group ID not allowed")
+		return bussGroupIds, nil
 	}
 
 	if bgid > 0 {
-		// Pass filter parameters, priority to use
 		return []int64{bgid}, nil
 	}
 
-	return bussGroupIds, nil
+	return bgids, nil
 }

center/router/router_datasource.go

@@ -148,11 +148,12 @@ func DatasourceCheck(ds models.Datasource) error {
 		},
 	}
 
+	ds.HTTPJson.Url = strings.TrimRight(ds.HTTPJson.Url, "/")
 	var fullURL string
 	req, err := ds.HTTPJson.NewReq(&fullURL)
 	if err != nil {
 		logger.Errorf("Error creating request: %v", err)
-		return fmt.Errorf("request urls:%v failed", ds.HTTPJson.GetUrls())
+		return fmt.Errorf("request urls:%v failed: %v", ds.HTTPJson.GetUrls(), err)
 	}
 
 	if ds.PluginType == models.PROMETHEUS {
@@ -168,14 +169,14 @@ func DatasourceCheck(ds models.Datasource) error {
 		req, err = http.NewRequest("GET", fullURL, nil)
 		if err != nil {
 			logger.Errorf("Error creating request: %v", err)
-			return fmt.Errorf("request url:%s failed", fullURL)
+			return fmt.Errorf("request url:%s failed: %v", fullURL, err)
 		}
 	} else if ds.PluginType == models.TDENGINE {
 		fullURL = fmt.Sprintf("%s/rest/sql", ds.HTTPJson.Url)
 		req, err = http.NewRequest("POST", fullURL, strings.NewReader("show databases"))
 		if err != nil {
 			logger.Errorf("Error creating request: %v", err)
-			return fmt.Errorf("request url:%s failed", fullURL)
+			return fmt.Errorf("request url:%s failed: %v", fullURL, err)
 		}
 	}
 
@@ -187,7 +188,7 @@ func DatasourceCheck(ds models.Datasource) error {
 		req, err = http.NewRequest("GET", fullURL, nil)
 		if err != nil {
 			logger.Errorf("Error creating request: %v", err)
-			return fmt.Errorf("request url:%s failed", fullURL)
+			return fmt.Errorf("request url:%s failed: %v", fullURL, err)
 		}
 	}
 
@@ -202,7 +203,7 @@ func DatasourceCheck(ds models.Datasource) error {
 	resp, err := client.Do(req)
 	if err != nil {
 		logger.Errorf("Error making request: %v\n", err)
-		return fmt.Errorf("request url:%s failed", fullURL)
+		return fmt.Errorf("request url:%s failed: %v", fullURL, err)
 	}
 	defer resp.Body.Close()
 

center/router/router_event_pipeline.go

@@ -4,8 +4,8 @@ import (
 	"net/http"
 	"time"
 
-	"github.com/ccfos/nightingale/v6/alert/pipeline"
 	"github.com/ccfos/nightingale/v6/models"
+
 	"github.com/gin-gonic/gin"
 	"github.com/toolkits/pkg/ginx"
 )
@@ -139,12 +139,23 @@ func (rt *Router) tryRunEventPipeline(c *gin.Context) {
 	}
 	event := hisEvent.ToCur()
 
-	for _, p := range f.PipelineConfig.Processors {
-		processor, err := pipeline.GetProcessorByType(p.Typ, p.Config)
+	for _, p := range f.PipelineConfig.ProcessorConfigs {
+		processor, err := models.GetProcessorByType(p.Typ, p.Config)
 		if err != nil {
-			ginx.Bomb(http.StatusBadRequest, "processor %+v type not found", p)
+			ginx.Bomb(http.StatusBadRequest, "get processor: %+v err: %+v", p, err)
+		}
+		event, _, err = processor.Process(rt.Ctx, event)
+		if err != nil {
+			ginx.Bomb(http.StatusBadRequest, "processor: %+v err: %+v", p, err)
+		}
+
+		if event == nil {
+			ginx.NewRender(c).Data(map[string]interface{}{
+				"event":  event,
+				"result": "event is dropped",
+			}, nil)
+			return
 		}
-		processor.Process(rt.Ctx, event)
 	}
 
 	ginx.NewRender(c).Data(event, nil)
@@ -153,8 +164,8 @@ func (rt *Router) tryRunEventPipeline(c *gin.Context) {
 // 测试事件处理器
 func (rt *Router) tryRunEventProcessor(c *gin.Context) {
 	var f struct {
-		EventId         int64            `json:"event_id"`
-		ProcessorConfig models.Processor `json:"processor_config"`
+		EventId         int64                  `json:"event_id"`
+		ProcessorConfig models.ProcessorConfig `json:"processor_config"`
 	}
 	ginx.BindJSON(c, &f)
 
@@ -164,13 +175,19 @@ func (rt *Router) tryRunEventProcessor(c *gin.Context) {
 	}
 	event := hisEvent.ToCur()
 
-	processor, err := pipeline.GetProcessorByType(f.ProcessorConfig.Typ, f.ProcessorConfig.Config)
+	processor, err := models.GetProcessorByType(f.ProcessorConfig.Typ, f.ProcessorConfig.Config)
 	if err != nil {
-		ginx.Bomb(http.StatusBadRequest, "processor type not found")
+		ginx.Bomb(http.StatusBadRequest, "get processor err: %+v", err)
+	}
+	event, res, err := processor.Process(rt.Ctx, event)
+	if err != nil {
+		ginx.Bomb(http.StatusBadRequest, "processor err: %+v", err)
 	}
-	processor.Process(rt.Ctx, event)
 
-	ginx.NewRender(c).Data(event, nil)
+	ginx.NewRender(c).Data(map[string]interface{}{
+		"event":  event,
+		"result": res,
+	}, nil)
 }
 
 func (rt *Router) tryRunEventProcessorByNotifyRule(c *gin.Context) {
@@ -199,12 +216,23 @@ func (rt *Router) tryRunEventProcessorByNotifyRule(c *gin.Context) {
 	}
 
 	for _, pl := range pipelines {
-		for _, p := range pl.Processors {
-			processor, err := pipeline.GetProcessorByType(p.Typ, p.Config)
+		for _, p := range pl.ProcessorConfigs {
+			processor, err := models.GetProcessorByType(p.Typ, p.Config)
 			if err != nil {
-				ginx.Bomb(http.StatusBadRequest, "processor %+v type not found", p)
+				ginx.Bomb(http.StatusBadRequest, "get processor: %+v err: %+v", p, err)
+			}
+
+			event, _, err := processor.Process(rt.Ctx, event)
+			if err != nil {
+				ginx.Bomb(http.StatusBadRequest, "processor: %+v err: %+v", p, err)
+			}
+			if event == nil {
+				ginx.NewRender(c).Data(map[string]interface{}{
+					"event":  event,
+					"result": "event is dropped",
+				}, nil)
+				return
 			}
-			processor.Process(rt.Ctx, event)
 		}
 	}
 

center/router/router_funcs.go

@@ -173,3 +173,38 @@ func Username(c *gin.Context) string {
 	}
 	return username
 }
+
+func HasPermission(ctx *ctx.Context, c *gin.Context, sourceType, sourceId string, isAnonymousAccess bool) bool {
+	if sourceType == "event" && isAnonymousAccess {
+		return true
+	}
+
+	// 尝试从请求中获取 __token 参数
+	token := ginx.QueryStr(c, "__token", "")
+
+	// 如果有 __token 参数，验证其合法性
+	if token != "" {
+		return ValidateSourceToken(ctx, sourceType, sourceId, token)
+	}
+
+	return false
+}
+
+func ValidateSourceToken(ctx *ctx.Context, sourceType, sourceId, token string) bool {
+	if token == "" {
+		return false
+	}
+
+	// 根据源类型、源ID和令牌获取源令牌记录
+	sourceToken, err := models.GetSourceTokenBySource(ctx, sourceType, sourceId, token)
+	if err != nil {
+		return false
+	}
+
+	// 检查令牌是否过期
+	if sourceToken.IsExpired() {
+		return false
+	}
+
+	return true
+}

center/router/router_message_template.go

@@ -12,7 +12,9 @@ import (
 	"github.com/ccfos/nightingale/v6/pkg/slice"
 	"github.com/ccfos/nightingale/v6/pkg/strx"
 	"github.com/ccfos/nightingale/v6/pkg/tplx"
+
 	"github.com/gin-gonic/gin"
+	"github.com/google/uuid"
 	"github.com/toolkits/pkg/ginx"
 )
 
@@ -30,6 +32,9 @@ func (rt *Router) messageTemplatesAdd(c *gin.Context) {
 	ginx.Dangerous(err)
 	now := time.Now().Unix()
 	for _, tpl := range lst {
+		// 生成一个唯一的标识符，以后也不允许修改，前端不需要传这个参数
+		tpl.Ident = uuid.New().String()
+
 		ginx.Dangerous(tpl.Verify())
 		if !isAdmin && !slice.HaveIntersection(gids, tpl.UserGroupIds) {
 			ginx.Bomb(http.StatusForbidden, "forbidden")

center/router/router_mute.go

@@ -65,6 +65,7 @@ func (rt *Router) alertMuteAdd(c *gin.Context) {
 
 	username := c.MustGet("username").(string)
 	f.CreateBy = username
+	f.UpdateBy = username
 	f.GroupId = ginx.UrlParamInt64(c, "id")
 	ginx.NewRender(c).Message(f.Add(rt.Ctx))
 }

center/router/router_notify_rule.go

@@ -237,6 +237,9 @@ func (rt *Router) notifyTest(c *gin.Context) {
 		}
 
 	case "smtp":
+		if len(sendtos) == 0 {
+			ginx.Bomb(http.StatusBadRequest, "No valid email address in the user and team")
+		}
 		err := notifyChannel.SendEmailNow(events, tplContent, sendtos)
 		ginx.NewRender(c).Message(err)
 	case "script":

center/router/router_source_token.go

@@ -0,0 +1,36 @@
+package router
+
+import (
+	"net/http"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/google/uuid"
+
+	"github.com/gin-gonic/gin"
+	"github.com/toolkits/pkg/ginx"
+)
+
+// sourceTokenAdd 生成新的源令牌
+func (rt *Router) sourceTokenAdd(c *gin.Context) {
+	var f models.SourceToken
+	ginx.BindJSON(c, &f)
+
+	if f.ExpireAt > 0 && f.ExpireAt <= time.Now().Unix() {
+		ginx.Bomb(http.StatusBadRequest, "expire time must be in the future")
+	}
+
+	token := uuid.New().String()
+
+	username := c.MustGet("username").(string)
+
+	f.Token = token
+	f.CreateBy = username
+	f.CreateAt = time.Now().Unix()
+
+	err := f.Add(rt.Ctx)
+	ginx.Dangerous(err)
+
+	go models.CleanupExpiredTokens(rt.Ctx)
+	ginx.NewRender(c).Data(token, nil)
+}

datasource/datasource.go

@@ -53,6 +53,20 @@ func init() {
 		PluginType:     "ck",
 		PluginTypeName: "ClickHouse",
 	}
+
+	DatasourceTypes[5] = DatasourceType{
+		Id:             5,
+		Category:       "timeseries",
+		PluginType:     "mysql",
+		PluginTypeName: "MySQL",
+	}
+
+	DatasourceTypes[6] = DatasourceType{
+		Id:             6,
+		Category:       "timeseries",
+		PluginType:     "pgsql",
+		PluginTypeName: "PostgreSQL",
+	}
 }
 
 type NewDatasrouceFn func(settings map[string]interface{}) (Datasource, error)

datasource/mysql/mysql.go

@@ -0,0 +1,227 @@
+package mysql
+
+import (
+	"context"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/datasource"
+	"github.com/ccfos/nightingale/v6/dskit/mysql"
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/ccfos/nightingale/v6/pkg/macros"
+
+	"github.com/mitchellh/mapstructure"
+	"github.com/toolkits/pkg/logger"
+)
+
+const (
+	MySQLType = "mysql"
+)
+
+func init() {
+	datasource.RegisterDatasource(MySQLType, new(MySQL))
+}
+
+type MySQL struct {
+	mysql.MySQL `json:",inline" mapstructure:",squash"`
+}
+
+type QueryParam struct {
+	Ref      string          `json:"ref" mapstructure:"ref"`
+	Database string          `json:"database" mapstructure:"database"`
+	Table    string          `json:"table" mapstructure:"table"`
+	SQL      string          `json:"sql" mapstructure:"sql"`
+	Keys     datasource.Keys `json:"keys" mapstructure:"keys"`
+	From     int64           `json:"from" mapstructure:"from"`
+	To       int64           `json:"to" mapstructure:"to"`
+}
+
+func (m *MySQL) InitClient() error {
+	if len(m.Shards) == 0 {
+		return fmt.Errorf("not found mysql addr, please check datasource config")
+	}
+	if _, err := m.NewConn(context.TODO(), ""); err != nil {
+		return err
+	}
+	return nil
+}
+
+func (m *MySQL) Init(settings map[string]interface{}) (datasource.Datasource, error) {
+	newest := new(MySQL)
+	err := mapstructure.Decode(settings, newest)
+	return newest, err
+}
+
+func (m *MySQL) Validate(ctx context.Context) error {
+	if len(m.Shards) == 0 || len(strings.TrimSpace(m.Shards[0].Addr)) == 0 {
+		return fmt.Errorf("mysql addr is invalid, please check datasource setting")
+	}
+
+	if len(strings.TrimSpace(m.Shards[0].User)) == 0 {
+		return fmt.Errorf("mysql user is invalid, please check datasource setting")
+	}
+
+	return nil
+}
+
+// Equal compares whether two objects are the same, used for caching
+func (m *MySQL) Equal(p datasource.Datasource) bool {
+	newest, ok := p.(*MySQL)
+	if !ok {
+		logger.Errorf("unexpected plugin type, expected is mysql")
+		return false
+	}
+
+	if len(m.Shards) == 0 || len(newest.Shards) == 0 {
+		return false
+	}
+
+	oldShard := m.Shards[0]
+	newShard := newest.Shards[0]
+
+	if oldShard.Addr != newShard.Addr {
+		return false
+	}
+
+	if oldShard.User != newShard.User {
+		return false
+	}
+
+	if oldShard.Password != newShard.Password {
+		return false
+	}
+
+	if oldShard.MaxQueryRows != newShard.MaxQueryRows {
+		return false
+	}
+
+	if oldShard.Timeout != newShard.Timeout {
+		return false
+	}
+
+	if oldShard.MaxIdleConns != newShard.MaxIdleConns {
+		return false
+	}
+
+	if oldShard.MaxOpenConns != newShard.MaxOpenConns {
+		return false
+	}
+
+	if oldShard.ConnMaxLifetime != newShard.ConnMaxLifetime {
+		return false
+	}
+
+	return true
+}
+
+func (m *MySQL) MakeLogQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
+	return nil, nil
+}
+
+func (m *MySQL) MakeTSQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
+	return nil, nil
+}
+
+func (m *MySQL) QueryMapData(ctx context.Context, query interface{}) ([]map[string]string, error) {
+	return nil, nil
+}
+
+func (m *MySQL) QueryData(ctx context.Context, query interface{}) ([]models.DataResp, error) {
+	mysqlQueryParam := new(QueryParam)
+	if err := mapstructure.Decode(query, mysqlQueryParam); err != nil {
+		return nil, err
+	}
+
+	if strings.Contains(mysqlQueryParam.SQL, "$__") {
+		var err error
+		mysqlQueryParam.SQL, err = macros.Macro(mysqlQueryParam.SQL, mysqlQueryParam.From, mysqlQueryParam.To)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	if mysqlQueryParam.Keys.ValueKey == "" {
+		return nil, fmt.Errorf("valueKey is required")
+	}
+
+	timeout := m.Shards[0].Timeout
+	if timeout == 0 {
+		timeout = 60
+	}
+
+	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
+	defer cancel()
+
+	items, err := m.QueryTimeseries(timeoutCtx, &sqlbase.QueryParam{
+		Sql: mysqlQueryParam.SQL,
+		Keys: types.Keys{
+			ValueKey: mysqlQueryParam.Keys.ValueKey,
+			LabelKey: mysqlQueryParam.Keys.LabelKey,
+			TimeKey:  mysqlQueryParam.Keys.TimeKey,
+		},
+	})
+
+	if err != nil {
+		logger.Warningf("query:%+v get data err:%v", mysqlQueryParam, err)
+		return []models.DataResp{}, err
+	}
+	data := make([]models.DataResp, 0)
+	for i := range items {
+		data = append(data, models.DataResp{
+			Ref:    mysqlQueryParam.Ref,
+			Metric: items[i].Metric,
+			Values: items[i].Values,
+		})
+	}
+
+	return data, nil
+}
+
+func (m *MySQL) QueryLog(ctx context.Context, query interface{}) ([]interface{}, int64, error) {
+	mysqlQueryParam := new(QueryParam)
+	if err := mapstructure.Decode(query, mysqlQueryParam); err != nil {
+		return nil, 0, err
+	}
+
+	if strings.Contains(mysqlQueryParam.SQL, "$__") {
+		var err error
+		mysqlQueryParam.SQL, err = macros.Macro(mysqlQueryParam.SQL, mysqlQueryParam.From, mysqlQueryParam.To)
+		if err != nil {
+			return nil, 0, err
+		}
+	}
+
+	timeout := m.Shards[0].Timeout
+	if timeout == 0 {
+		timeout = 60
+	}
+
+	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
+	defer cancel()
+
+	items, err := m.Query(timeoutCtx, &sqlbase.QueryParam{
+		Sql: mysqlQueryParam.SQL,
+	})
+
+	if err != nil {
+		logger.Warningf("query:%+v get data err:%v", mysqlQueryParam, err)
+		return []interface{}{}, 0, err
+	}
+	logs := make([]interface{}, 0)
+	for i := range items {
+		logs = append(logs, items[i])
+	}
+
+	return logs, 0, nil
+}
+
+func (m *MySQL) DescribeTable(ctx context.Context, query interface{}) ([]*types.ColumnProperty, error) {
+	mysqlQueryParam := new(QueryParam)
+	if err := mapstructure.Decode(query, mysqlQueryParam); err != nil {
+		return nil, err
+	}
+	return m.DescTable(ctx, mysqlQueryParam.Database, mysqlQueryParam.Table)
+}

datasource/postgresql/postgresql.go

@@ -0,0 +1,346 @@
+package postgresql
+
+import (
+	"context"
+	"fmt"
+	"regexp"
+	"strings"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/datasource"
+	"github.com/ccfos/nightingale/v6/pkg/macros"
+
+	"github.com/ccfos/nightingale/v6/dskit/postgres"
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+	"github.com/ccfos/nightingale/v6/models"
+	"github.com/mitchellh/mapstructure"
+	"github.com/toolkits/pkg/logger"
+)
+
+const (
+	PostgreSQLType = "pgsql"
+)
+
+var (
+	regx = "(?i)from\\s+([a-zA-Z0-9_]+)\\.([a-zA-Z0-9_]+)\\.([a-zA-Z0-9_]+)"
+)
+
+func init() {
+	datasource.RegisterDatasource(PostgreSQLType, new(PostgreSQL))
+}
+
+type PostgreSQL struct {
+	Shards []*postgres.PostgreSQL `json:"pgsql.shards" mapstructure:"pgsql.shards"`
+}
+
+type QueryParam struct {
+	Ref      string          `json:"ref" mapstructure:"ref"`
+	Database string          `json:"database" mapstructure:"database"`
+	Table    string          `json:"table" mapstructure:"table"`
+	SQL      string          `json:"sql" mapstructure:"sql"`
+	Keys     datasource.Keys `json:"keys" mapstructure:"keys"`
+	From     int64           `json:"from" mapstructure:"from"`
+	To       int64           `json:"to" mapstructure:"to"`
+}
+
+func (p *PostgreSQL) InitClient() error {
+	if len(p.Shards) == 0 {
+		return fmt.Errorf("not found postgresql addr, please check datasource config")
+	}
+	for _, shard := range p.Shards {
+		if db, err := shard.NewConn(context.TODO(), "postgres"); err != nil {
+			defer sqlbase.CloseDB(db)
+			return err
+		}
+	}
+	return nil
+}
+
+func (p *PostgreSQL) Init(settings map[string]interface{}) (datasource.Datasource, error) {
+	newest := new(PostgreSQL)
+	err := mapstructure.Decode(settings, newest)
+	return newest, err
+}
+
+func (p *PostgreSQL) Validate(ctx context.Context) error {
+	if len(p.Shards) == 0 || len(strings.TrimSpace(p.Shards[0].Addr)) == 0 {
+		return fmt.Errorf("postgresql addr is invalid, please check datasource setting")
+	}
+
+	if len(strings.TrimSpace(p.Shards[0].User)) == 0 {
+		return fmt.Errorf("postgresql user is invalid, please check datasource setting")
+	}
+
+	return nil
+}
+
+// Equal compares whether two objects are the same, used for caching
+func (p *PostgreSQL) Equal(d datasource.Datasource) bool {
+	newest, ok := d.(*PostgreSQL)
+	if !ok {
+		logger.Errorf("unexpected plugin type, expected is postgresql")
+		return false
+	}
+
+	if len(p.Shards) == 0 || len(newest.Shards) == 0 {
+		return false
+	}
+
+	oldShard := p.Shards[0]
+	newShard := newest.Shards[0]
+
+	if oldShard.Addr != newShard.Addr {
+		return false
+	}
+
+	if oldShard.User != newShard.User {
+		return false
+	}
+
+	if oldShard.Password != newShard.Password {
+		return false
+	}
+
+	if oldShard.MaxQueryRows != newShard.MaxQueryRows {
+		return false
+	}
+
+	if oldShard.Timeout != newShard.Timeout {
+		return false
+	}
+
+	if oldShard.MaxIdleConns != newShard.MaxIdleConns {
+		return false
+	}
+
+	if oldShard.MaxOpenConns != newShard.MaxOpenConns {
+		return false
+	}
+
+	if oldShard.ConnMaxLifetime != newShard.ConnMaxLifetime {
+		return false
+	}
+
+	return true
+}
+
+func (p *PostgreSQL) ShowDatabases(ctx context.Context) ([]string, error) {
+	return p.Shards[0].ShowDatabases(ctx, "")
+}
+
+func (p *PostgreSQL) ShowTables(ctx context.Context, database string) ([]string, error) {
+	p.Shards[0].DB = database
+	rets, err := p.Shards[0].ShowTables(ctx, "")
+	if err != nil {
+		return nil, err
+	}
+	tables := make([]string, 0, len(rets))
+	for scheme, tabs := range rets {
+		for _, tab := range tabs {
+			tables = append(tables, scheme+"."+tab)
+		}
+	}
+	return tables, nil
+}
+
+func (p *PostgreSQL) MakeLogQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
+	return nil, nil
+}
+
+func (p *PostgreSQL) MakeTSQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
+	return nil, nil
+}
+
+func (p *PostgreSQL) QueryMapData(ctx context.Context, query interface{}) ([]map[string]string, error) {
+	return nil, nil
+}
+
+func (p *PostgreSQL) QueryData(ctx context.Context, query interface{}) ([]models.DataResp, error) {
+	postgresqlQueryParam := new(QueryParam)
+	if err := mapstructure.Decode(query, postgresqlQueryParam); err != nil {
+		return nil, err
+	}
+
+	if strings.Contains(postgresqlQueryParam.SQL, "$__") {
+		var err error
+		postgresqlQueryParam.SQL, err = macros.Macro(postgresqlQueryParam.SQL, postgresqlQueryParam.From, postgresqlQueryParam.To)
+		if err != nil {
+			return nil, err
+		}
+	}
+	if postgresqlQueryParam.Database != "" {
+		p.Shards[0].DB = postgresqlQueryParam.Database
+	} else {
+		db, err := parseDBName(postgresqlQueryParam.SQL)
+		if err != nil {
+			return nil, err
+		}
+		p.Shards[0].DB = db
+	}
+
+	timeout := p.Shards[0].Timeout
+	if timeout == 0 {
+		timeout = 60
+	}
+	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
+	defer cancel()
+
+	items, err := p.Shards[0].QueryTimeseries(timeoutCtx, &sqlbase.QueryParam{
+		Sql: postgresqlQueryParam.SQL,
+		Keys: types.Keys{
+			ValueKey: postgresqlQueryParam.Keys.ValueKey,
+			LabelKey: postgresqlQueryParam.Keys.LabelKey,
+			TimeKey:  postgresqlQueryParam.Keys.TimeKey,
+		},
+	})
+
+	if err != nil {
+		logger.Warningf("query:%+v get data err:%v", postgresqlQueryParam, err)
+		return []models.DataResp{}, err
+	}
+	data := make([]models.DataResp, 0)
+	for i := range items {
+		data = append(data, models.DataResp{
+			Ref:    postgresqlQueryParam.Ref,
+			Metric: items[i].Metric,
+			Values: items[i].Values,
+		})
+	}
+
+	// parse resp to time series data
+	logger.Infof("req:%+v keys:%+v \n data:%v", postgresqlQueryParam, postgresqlQueryParam.Keys, data)
+
+	return data, nil
+}
+
+func (p *PostgreSQL) QueryLog(ctx context.Context, query interface{}) ([]interface{}, int64, error) {
+	postgresqlQueryParam := new(QueryParam)
+	if err := mapstructure.Decode(query, postgresqlQueryParam); err != nil {
+		return nil, 0, err
+	}
+	if postgresqlQueryParam.Database != "" {
+		p.Shards[0].DB = postgresqlQueryParam.Database
+	} else {
+		db, err := parseDBName(postgresqlQueryParam.SQL)
+		if err != nil {
+			return nil, 0, err
+		}
+		p.Shards[0].DB = db
+	}
+
+	if strings.Contains(postgresqlQueryParam.SQL, "$__") {
+		var err error
+		postgresqlQueryParam.SQL, err = macros.Macro(postgresqlQueryParam.SQL, postgresqlQueryParam.From, postgresqlQueryParam.To)
+		if err != nil {
+			return nil, 0, err
+		}
+	}
+
+	timeout := p.Shards[0].Timeout
+	if timeout == 0 {
+		timeout = 60
+	}
+	timeoutCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
+	defer cancel()
+	items, err := p.Shards[0].Query(timeoutCtx, &sqlbase.QueryParam{
+		Sql: postgresqlQueryParam.SQL,
+	})
+	if err != nil {
+		logger.Warningf("query:%+v get data err:%v", postgresqlQueryParam, err)
+		return []interface{}{}, 0, err
+	}
+	logs := make([]interface{}, 0)
+	for i := range items {
+		logs = append(logs, items[i])
+	}
+
+	return logs, 0, nil
+}
+
+func (p *PostgreSQL) DescribeTable(ctx context.Context, query interface{}) ([]*types.ColumnProperty, error) {
+	postgresqlQueryParam := new(QueryParam)
+	if err := mapstructure.Decode(query, postgresqlQueryParam); err != nil {
+		return nil, err
+	}
+	p.Shards[0].DB = postgresqlQueryParam.Database
+	pairs := strings.Split(postgresqlQueryParam.Table, ".") // format: scheme.table_name
+	scheme := ""
+	table := postgresqlQueryParam.Table
+	if len(pairs) == 2 {
+		scheme = pairs[0]
+		table = pairs[1]
+	}
+	return p.Shards[0].DescTable(ctx, scheme, table)
+}
+
+func parseDBName(sql string) (db string, err error) {
+	re := regexp.MustCompile(regx)
+	matches := re.FindStringSubmatch(sql)
+	if len(matches) != 4 {
+		return "", fmt.Errorf("no valid table name in format database.schema.table found")
+	}
+	return matches[1], nil
+}
+
+func extractColumns(sql string) ([]string, error) {
+	// 将 SQL 转换为小写以简化匹配
+	sql = strings.ToLower(sql)
+
+	// 匹配 SELECT 和 FROM 之间的内容
+	re := regexp.MustCompile(`select\s+(.*?)\s+from`)
+	matches := re.FindStringSubmatch(sql)
+
+	if len(matches) < 2 {
+		return nil, fmt.Errorf("no columns found or invalid SQL syntax")
+	}
+
+	// 提取列部分
+	columnsString := matches[1]
+
+	// 分割列
+	columns := splitColumns(columnsString)
+
+	// 清理每个列名
+	for i, col := range columns {
+		columns[i] = strings.TrimSpace(col)
+	}
+
+	return columns, nil
+}
+
+func splitColumns(columnsString string) []string {
+	var columns []string
+	var currentColumn strings.Builder
+	parenthesesCount := 0
+	inQuotes := false
+
+	for _, char := range columnsString {
+		switch char {
+		case '(':
+			parenthesesCount++
+			currentColumn.WriteRune(char)
+		case ')':
+			parenthesesCount--
+			currentColumn.WriteRune(char)
+		case '\'', '"':
+			inQuotes = !inQuotes
+			currentColumn.WriteRune(char)
+		case ',':
+			if parenthesesCount == 0 && !inQuotes {
+				columns = append(columns, currentColumn.String())
+				currentColumn.Reset()
+			} else {
+				currentColumn.WriteRune(char)
+			}
+		default:
+			currentColumn.WriteRune(char)
+		}
+	}
+
+	if currentColumn.Len() > 0 {
+		columns = append(columns, currentColumn.String())
+	}
+
+	return columns
+}

docker/compose-postgres/initsql_for_postgres/a-n9e-for-Postgres.sql

@@ -204,6 +204,7 @@ CREATE TABLE board (
     public smallint not null default 0 ,
     built_in smallint not null default 0 ,
     hide smallint not null default 0 ,
+    public_cate bigint NOT NULL DEFAULT 0,
     create_at bigint not null default 0,
     create_by varchar(64) not null default '',
     update_at bigint not null default 0,
@@ -217,6 +218,7 @@ COMMENT ON COLUMN board.tags IS 'split by space';
 COMMENT ON COLUMN board.public IS '0:false 1:true';
 COMMENT ON COLUMN board.built_in IS '0:false 1:true';
 COMMENT ON COLUMN board.hide IS '0:false 1:true';
+COMMENT ON COLUMN board.public_cate IS '0 anonymous 1 login 2 busi';
 
 
 -- for dashboard new version
@@ -429,43 +431,31 @@ CREATE TABLE target (
     ident varchar(191) not null,
     note varchar(255) not null default '',
     tags varchar(512) not null default '',
+    host_tags text,
     host_ip varchar(15) default '',
     agent_version varchar(255) default '',
     engine_name varchar(255) default '',
+    os varchar(31) default '',
     update_at bigint not null default 0,
     PRIMARY KEY (id),
     UNIQUE (ident)
 );
 
 CREATE INDEX ON target (group_id);
+CREATE INDEX idx_host_ip ON target (host_ip);
+CREATE INDEX idx_agent_version ON target (agent_version);
+CREATE INDEX idx_engine_name ON target (engine_name);
+CREATE INDEX idx_os ON target (os);
 
 COMMENT ON COLUMN target.group_id IS 'busi group id';
 COMMENT ON COLUMN target.ident IS 'target id';
 COMMENT ON COLUMN target.note IS 'append to alert event as field';
 COMMENT ON COLUMN target.tags IS 'append to series data as tags, split by space, append external space at suffix';
+COMMENT ON COLUMN target.host_tags IS 'global labels set in conf file';
 COMMENT ON COLUMN target.host_ip IS 'IPv4 string';
 COMMENT ON COLUMN target.agent_version IS 'agent version';
 COMMENT ON COLUMN target.engine_name IS 'engine_name';
--- case1: target_idents; case2: target_tags
--- CREATE TABLE collect_rule (
---     id bigserial,
---     group_id bigint not null default 0 comment 'busi group id',
---     cluster varchar(128) not null,
---     target_idents varchar(512) not null default '' comment 'ident list, split by space',
---     target_tags varchar(512) not null default '' comment 'filter targets by tags, split by space',
---     name varchar(191) not null default '',
---     note varchar(255) not null default '',
---     step int not null,
---     type varchar(64) not null comment 'e.g. port proc log plugin',
---     data text not null,
---     append_tags varchar(255) not null default '' comment 'split by space: e.g. mod=n9e dept=cloud',
---     create_at bigint not null default 0,
---     create_by varchar(64) not null default '',
---     update_at bigint not null default 0,
---     update_by varchar(64) not null default '',
---     PRIMARY KEY (id),
---     KEY (group_id, type, name)
--- ) ;
+COMMENT ON COLUMN target.os IS 'os type';
 
 CREATE TABLE metric_view (
     id bigserial,
@@ -519,7 +509,6 @@ CREATE TABLE alert_aggr_view (
     id bigserial,
     name varchar(191) not null default '',
     rule varchar(2048) not null default '',
-    format varchar(2048) not null default '',
     cate smallint not null ,
     create_at bigint not null default 0,
     create_by bigint not null default 0,
@@ -735,6 +724,7 @@ CREATE TABLE datasource
 (
     id serial,
     name varchar(191) not null default '',
+    identifier varchar(255) not null default '',
     description varchar(255) not null default '',
     category varchar(255) not null default '',
     plugin_id int  not null default 0,
@@ -752,8 +742,8 @@ CREATE TABLE datasource
     updated_by varchar(64) not null default '',
     UNIQUE (name),
     PRIMARY KEY (id)
-) ; 
-  
+) ;
+
 CREATE TABLE builtin_cate (
     id bigserial,
     name varchar(191) not null,
@@ -796,10 +786,12 @@ CREATE TABLE es_index_pattern (
     create_by varchar(64) default '',
     update_at bigint default '0',
     update_by varchar(64) default '',
+    note varchar(4096) not null default '',
     PRIMARY KEY (id),
     UNIQUE (datasource_id, name)
 ) ;
 COMMENT ON COLUMN es_index_pattern.datasource_id IS 'datasource id';
+COMMENT ON COLUMN es_index_pattern.note IS 'description of metric in Chinese';
 
 CREATE TABLE builtin_metrics (
   id bigserial,
@@ -814,6 +806,7 @@ CREATE TABLE builtin_metrics (
   created_by varchar(191) NOT NULL DEFAULT '',
   updated_at bigint NOT NULL DEFAULT 0,
   updated_by varchar(191) NOT NULL DEFAULT '',
+  uuid BIGINT NOT NULL DEFAULT 0,
   PRIMARY KEY (id),
   UNIQUE (lang, collector, typ, name)
 );
@@ -835,6 +828,7 @@ COMMENT ON COLUMN builtin_metrics.created_at IS 'create time';
 COMMENT ON COLUMN builtin_metrics.created_by IS 'creator';
 COMMENT ON COLUMN builtin_metrics.updated_at IS 'update time';
 COMMENT ON COLUMN builtin_metrics.updated_by IS 'updater';
+COMMENT ON COLUMN builtin_metrics.uuid IS 'unique identifier';
 
 CREATE TABLE metric_filter (
   id BIGSERIAL PRIMARY KEY,
@@ -904,4 +898,128 @@ CREATE TABLE dash_annotation (
     create_by varchar(64) not null default '',
     update_at bigint not null default 0,
     update_by varchar(64) not null default ''
+);
+
+CREATE TABLE source_token (
+    id bigserial PRIMARY KEY,
+    source_type varchar(64) NOT NULL DEFAULT '',
+    source_id varchar(255) NOT NULL DEFAULT '',
+    token varchar(255) NOT NULL DEFAULT '',
+    expire_at bigint NOT NULL DEFAULT 0,
+    create_at bigint NOT NULL DEFAULT 0,
+    create_by varchar(64) NOT NULL DEFAULT ''
+);
+
+CREATE INDEX idx_source_token_type_id_token ON source_token (source_type, source_id, token);
+
+CREATE TABLE notification_record (
+    id BIGSERIAL PRIMARY KEY,
+    notify_rule_id BIGINT NOT NULL DEFAULT 0,
+    event_id bigint NOT NULL,
+    sub_id bigint DEFAULT NULL,
+    channel varchar(255) NOT NULL,
+    status bigint DEFAULT NULL,
+    target varchar(1024) NOT NULL,
+    details varchar(2048) DEFAULT '',
+    created_at bigint NOT NULL
+);
+
+CREATE INDEX idx_evt ON notification_record (event_id);
+
+COMMENT ON COLUMN notification_record.event_id IS 'event history id';
+COMMENT ON COLUMN notification_record.sub_id IS 'subscribed rule id';
+COMMENT ON COLUMN notification_record.channel IS 'notification channel name';
+COMMENT ON COLUMN notification_record.status IS 'notification status';
+COMMENT ON COLUMN notification_record.target IS 'notification target';
+COMMENT ON COLUMN notification_record.details IS 'notification other info';
+COMMENT ON COLUMN notification_record.created_at IS 'create time';
+
+CREATE TABLE target_busi_group (
+    id BIGSERIAL PRIMARY KEY,
+    target_ident varchar(191) NOT NULL,
+    group_id bigint NOT NULL,
+    update_at bigint NOT NULL
+);
+
+CREATE UNIQUE INDEX idx_target_group ON target_busi_group (target_ident, group_id);
+
+CREATE TABLE user_token (
+    id BIGSERIAL PRIMARY KEY,
+    username varchar(255) NOT NULL DEFAULT '',
+    token_name varchar(255) NOT NULL DEFAULT '',
+    token varchar(255) NOT NULL DEFAULT '',
+    create_at bigint NOT NULL DEFAULT 0,
+    last_used bigint NOT NULL DEFAULT 0
+);
+
+CREATE TABLE notify_rule (
+    id bigserial PRIMARY KEY,
+    name varchar(255) NOT NULL,
+    description text,
+    enable smallint NOT NULL DEFAULT 0,
+    user_group_ids varchar(255) NOT NULL DEFAULT '',
+    notify_configs text,
+    pipeline_configs text,
+    create_at bigint NOT NULL DEFAULT 0,
+    create_by varchar(64) NOT NULL DEFAULT '',
+    update_at bigint NOT NULL DEFAULT 0,
+    update_by varchar(64) NOT NULL DEFAULT ''
+);
+
+CREATE TABLE notify_channel (
+    id bigserial PRIMARY KEY,
+    name varchar(255) NOT NULL,
+    ident varchar(255) NOT NULL,
+    description text,
+    enable smallint NOT NULL DEFAULT 0,
+    param_config text,
+    request_type varchar(50) NOT NULL,
+    request_config text,
+    weight int NOT NULL DEFAULT 0,
+    create_at bigint NOT NULL DEFAULT 0,
+    create_by varchar(64) NOT NULL DEFAULT '',
+    update_at bigint NOT NULL DEFAULT 0,
+    update_by varchar(64) NOT NULL DEFAULT ''
+);
+
+CREATE TABLE message_template (
+    id bigserial PRIMARY KEY,
+    name varchar(64) NOT NULL,
+    ident varchar(64) NOT NULL,
+    content text,
+    user_group_ids varchar(64),
+    notify_channel_ident varchar(64) NOT NULL DEFAULT '',
+    private int NOT NULL DEFAULT 0,
+    weight int NOT NULL DEFAULT 0,
+    create_at bigint NOT NULL DEFAULT 0,
+    create_by varchar(64) NOT NULL DEFAULT '',
+    update_at bigint NOT NULL DEFAULT 0,
+    update_by varchar(64) NOT NULL DEFAULT ''
+);
+
+CREATE TABLE event_pipeline (
+    id bigserial PRIMARY KEY,
+    name varchar(128) NOT NULL,
+    team_ids text,
+    description varchar(255) NOT NULL DEFAULT '',
+    filter_enable smallint NOT NULL DEFAULT 0,
+    label_filters text,
+    attribute_filters text,
+    processors text,
+    create_at bigint NOT NULL DEFAULT 0,
+    create_by varchar(64) NOT NULL DEFAULT '',
+    update_at bigint NOT NULL DEFAULT 0,
+    update_by varchar(64) NOT NULL DEFAULT ''
+);
+
+CREATE TABLE embedded_product (
+    id bigserial PRIMARY KEY,
+    name varchar(255) DEFAULT NULL,
+    url varchar(255) DEFAULT NULL,
+    is_private boolean DEFAULT NULL,
+    team_ids varchar(255),
+    create_at bigint NOT NULL DEFAULT 0,
+    create_by varchar(64) NOT NULL DEFAULT '',
+    update_at bigint NOT NULL DEFAULT 0,
+    update_by varchar(64) NOT NULL DEFAULT ''
 );

docker/initsql/a-n9e.sql

@@ -421,7 +421,6 @@ CREATE TABLE `alert_aggr_view` (
     `id` bigint unsigned not null auto_increment,
     `name` varchar(191) not null default '',
     `rule` varchar(2048) not null default '',
-    `format` varchar(2048) not null default '',
     `cate` tinyint(1) not null comment '0: preset 1: custom',
     `create_at` bigint not null default 0,
     `create_by` bigint not null default 0 comment 'user id',
@@ -466,6 +465,7 @@ CREATE TABLE `alert_cur_event` (
     `rule_config` text not null comment 'annotations',
     `tags` varchar(1024) not null default '' comment 'merge data_tags rule_tags, split by ,,',
     `original_tags` text comment 'labels key=val,,k2=v2',
+    `notify_rule_ids` text COMMENT 'notify rule ids',
     PRIMARY KEY (`id`),
     KEY (`hash`),
     KEY (`rule_id`),
@@ -508,6 +508,7 @@ CREATE TABLE `alert_his_event` (
     `original_tags` text comment 'labels key=val,,k2=v2',
     `annotations` text not null comment 'annotations',
     `rule_config` text not null comment 'annotations',
+    `notify_rule_ids` text COMMENT 'notify rule ids',
     PRIMARY KEY (`id`),
     INDEX `idx_last_eval_time` (`last_eval_time`),
     KEY (`hash`),
@@ -2215,3 +2216,15 @@ CREATE TABLE task_host_99
     PRIMARY KEY (`ii`)
 ) ENGINE = InnoDB
   DEFAULT CHARSET = utf8mb4;
+
+CREATE TABLE `source_token` (
+    `id` bigint unsigned NOT NULL AUTO_INCREMENT,
+    `source_type` varchar(64) NOT NULL DEFAULT '' COMMENT 'source type',
+    `source_id` varchar(255) NOT NULL DEFAULT '' COMMENT 'source identifier',
+    `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'access token',
+    `expire_at` bigint NOT NULL DEFAULT 0 COMMENT 'expire timestamp',
+    `create_at` bigint NOT NULL DEFAULT 0 COMMENT 'create timestamp',
+    `create_by` varchar(64) NOT NULL DEFAULT '' COMMENT 'creator',
+    PRIMARY KEY (`id`),
+    KEY `idx_source_type_id_token` (`source_type`, `source_id`, `token`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

docker/migratesql/migrate.sql

@@ -246,3 +246,19 @@ CREATE TABLE `event_pipeline` (
     PRIMARY KEY (`id`)
 ) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4;
 
+/* v8.0.0-next */
+CREATE TABLE `source_token` (
+    `id` bigint unsigned NOT NULL AUTO_INCREMENT,
+    `source_type` varchar(64) NOT NULL DEFAULT '' COMMENT 'source type',
+    `source_id` varchar(255) NOT NULL DEFAULT '' COMMENT 'source identifier',
+    `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'access token',
+    `expire_at` bigint NOT NULL DEFAULT 0 COMMENT 'expire timestamp',
+    `create_at` bigint NOT NULL DEFAULT 0 COMMENT 'create timestamp',
+    `create_by` varchar(64) NOT NULL DEFAULT '' COMMENT 'creator',
+    PRIMARY KEY (`id`),
+    KEY `idx_source_type_id_token` (`source_type`, `source_id`, `token`)
+) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
+
+/* v8.0.0-beta.12 2025-06-03 */
+ALTER TABLE `alert_his_event` ADD COLUMN `notify_rule_ids` text COMMENT 'notify rule ids';
+ALTER TABLE `alert_cur_event` ADD COLUMN `notify_rule_ids` text COMMENT 'notify rule ids';

docker/sqlite.sql

@@ -374,7 +374,6 @@ CREATE TABLE `alert_aggr_view` (
     `id` integer primary key autoincrement,
     `name` varchar(191) not null default '',
     `rule` varchar(2048) not null default '',
-    `format` varchar(2048) not null default '',
     `cate` tinyint(1) not null,
     `create_at` bigint not null default 0,
     `create_by` bigint not null default 0,

dscache/sync.go

@@ -9,6 +9,8 @@ import (
 	"github.com/ccfos/nightingale/v6/datasource"
 	_ "github.com/ccfos/nightingale/v6/datasource/ck"
 	"github.com/ccfos/nightingale/v6/datasource/es"
+	_ "github.com/ccfos/nightingale/v6/datasource/mysql"
+	_ "github.com/ccfos/nightingale/v6/datasource/postgresql"
 	"github.com/ccfos/nightingale/v6/dskit/tdengine"
 	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/ctx"
@@ -191,7 +193,14 @@ func PutDatasources(items []datasource.DatasourceInfo) {
 		ids = append(ids, item.Id)
 
 		// 异步初始化 client 不然数据源同步的会很慢
-		go DsCache.Put(typ, item.Id, ds)
+		go func() {
+			defer func() {
+				if r := recover(); r != nil {
+					logger.Errorf("panic in datasource item: %+v panic:%v", item, r)
+				}
+			}()
+			DsCache.Put(typ, item.Id, ds)
+		}()
 	}
 
 	logger.Debugf("get plugin by type success Ids:%v", ids)

dskit/mysql/mysql.go

@@ -0,0 +1,172 @@
+// @Author: Ciusyan 5/10/24
+
+package mysql
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/dskit/pool"
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+
+	_ "github.com/go-sql-driver/mysql" // MySQL driver
+	"github.com/mitchellh/mapstructure"
+	"gorm.io/driver/mysql"
+	"gorm.io/gorm"
+)
+
+type MySQL struct {
+	Shards []Shard `json:"mysql.shards" mapstructure:"mysql.shards"`
+}
+
+type Shard struct {
+	Addr            string `json:"mysql.addr" mapstructure:"mysql.addr"`
+	DB              string `json:"mysql.db" mapstructure:"mysql.db"`
+	User            string `json:"mysql.user" mapstructure:"mysql.user"`
+	Password        string `json:"mysql.password" mapstructure:"mysql.password"`
+	Timeout         int    `json:"mysql.timeout" mapstructure:"mysql.timeout"`
+	MaxIdleConns    int    `json:"mysql.max_idle_conns" mapstructure:"mysql.max_idle_conns"`
+	MaxOpenConns    int    `json:"mysql.max_open_conns" mapstructure:"mysql.max_open_conns"`
+	ConnMaxLifetime int    `json:"mysql.conn_max_lifetime" mapstructure:"mysql.conn_max_lifetime"`
+	MaxQueryRows    int    `json:"mysql.max_query_rows" mapstructure:"mysql.max_query_rows"`
+}
+
+func NewMySQLWithSettings(ctx context.Context, settings interface{}) (*MySQL, error) {
+	newest := new(MySQL)
+	settingsMap := map[string]interface{}{}
+
+	switch s := settings.(type) {
+	case string:
+		if err := json.Unmarshal([]byte(s), &settingsMap); err != nil {
+			return nil, err
+		}
+	case map[string]interface{}:
+		settingsMap = s
+	default:
+		return nil, errors.New("unsupported settings type")
+	}
+
+	if err := mapstructure.Decode(settingsMap, newest); err != nil {
+		return nil, err
+	}
+
+	return newest, nil
+}
+
+// NewConn establishes a new connection to MySQL
+func (m *MySQL) NewConn(ctx context.Context, database string) (*gorm.DB, error) {
+	if len(m.Shards) == 0 {
+		return nil, errors.New("empty pgsql shards")
+	}
+
+	shard := m.Shards[0]
+
+	if shard.Timeout == 0 {
+		shard.Timeout = 300
+	}
+
+	if shard.MaxIdleConns == 0 {
+		shard.MaxIdleConns = 10
+	}
+
+	if shard.MaxOpenConns == 0 {
+		shard.MaxOpenConns = 100
+	}
+
+	if shard.ConnMaxLifetime == 0 {
+		shard.ConnMaxLifetime = 300
+	}
+
+	if shard.MaxQueryRows == 0 {
+		shard.MaxQueryRows = 100
+	}
+
+	if len(shard.Addr) == 0 {
+		return nil, errors.New("empty addr")
+	}
+
+	if len(shard.Addr) == 0 {
+		return nil, errors.New("empty addr")
+	}
+	var keys []string
+	var err error
+	keys = append(keys, shard.Addr)
+
+	keys = append(keys, shard.Password, shard.User)
+	if len(database) > 0 {
+		keys = append(keys, database)
+	}
+	cachedKey := strings.Join(keys, ":")
+	// cache conn with database
+	conn, ok := pool.PoolClient.Load(cachedKey)
+	if ok {
+		return conn.(*gorm.DB), nil
+	}
+	var db *gorm.DB
+	defer func() {
+		if db != nil && err == nil {
+			pool.PoolClient.Store(cachedKey, db)
+		}
+	}()
+
+	dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8&parseTime=True", shard.User, shard.Password, shard.Addr, database)
+
+	return sqlbase.NewDB(
+		ctx,
+		mysql.Open(dsn),
+		shard.MaxIdleConns,
+		shard.MaxOpenConns,
+		time.Duration(shard.ConnMaxLifetime)*time.Second,
+	)
+}
+
+func (m *MySQL) ShowDatabases(ctx context.Context) ([]string, error) {
+	db, err := m.NewConn(ctx, "")
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.ShowDatabases(ctx, db, "SHOW DATABASES")
+}
+
+func (m *MySQL) ShowTables(ctx context.Context, database string) ([]string, error) {
+	db, err := m.NewConn(ctx, database)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.ShowTables(ctx, db, "SHOW TABLES")
+}
+
+func (m *MySQL) DescTable(ctx context.Context, database, table string) ([]*types.ColumnProperty, error) {
+	db, err := m.NewConn(ctx, database)
+	if err != nil {
+		return nil, err
+	}
+
+	query := fmt.Sprintf("DESCRIBE %s", table)
+	return sqlbase.DescTable(ctx, db, query)
+}
+
+func (m *MySQL) SelectRows(ctx context.Context, database, table, query string) ([]map[string]interface{}, error) {
+	db, err := m.NewConn(ctx, database)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.SelectRows(ctx, db, table, query)
+}
+
+func (m *MySQL) ExecQuery(ctx context.Context, database string, sql string) ([]map[string]interface{}, error) {
+	db, err := m.NewConn(ctx, database)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.ExecQuery(ctx, db, sql)
+}

dskit/mysql/mysql_test.go

@@ -0,0 +1,129 @@
+// @Author: Ciusyan 5/11/24
+
+package mysql
+
+import (
+	"context"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+)
+
+func TestNewMySQLWithSettings(t *testing.T) {
+	tests := []struct {
+		name     string
+		settings interface{}
+		wantErr  bool
+	}{
+		{
+			name:     "valid string settings",
+			settings: `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`,
+			wantErr:  false,
+		},
+		{
+			name:     "invalid settings type",
+			settings: 12345,
+			wantErr:  true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got, err := NewMySQLWithSettings(context.Background(), tt.settings)
+			if (err != nil) != tt.wantErr {
+				t.Errorf("NewMySQLWithSettings() error = %v, wantErr %v", err, tt.wantErr)
+			}
+			t.Log(got)
+		})
+	}
+}
+
+func TestNewConn(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	tests := []struct {
+		name     string
+		database string
+		wantErr  bool
+	}{
+		{
+			name:     "valid connection",
+			database: "db1",
+			wantErr:  false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			_, err := mysql.NewConn(ctx, tt.database)
+			if (err != nil) != tt.wantErr {
+				t.Errorf("NewConn() error = %v, wantErr %v", err, tt.wantErr)
+				return
+			}
+		})
+	}
+}
+
+func TestShowDatabases(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	databases, err := mysql.ShowDatabases(ctx)
+	require.NoError(t, err)
+	t.Log(databases)
+}
+
+func TestShowTables(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	tables, err := mysql.ShowTables(ctx, "db1")
+	require.NoError(t, err)
+	t.Log(tables)
+}
+
+func TestDescTable(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	descTable, err := mysql.DescTable(ctx, "db1", "students")
+	require.NoError(t, err)
+	for _, desc := range descTable {
+		t.Logf("%+v", *desc)
+	}
+}
+
+func TestExecQuery(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	rows, err := mysql.ExecQuery(ctx, "db1", "SELECT * FROM students WHERE id = 10008")
+	require.NoError(t, err)
+	for _, row := range rows {
+		t.Log(row)
+	}
+}
+
+func TestSelectRows(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	rows, err := mysql.SelectRows(ctx, "db1", "students", "id > 10008")
+	require.NoError(t, err)
+	for _, row := range rows {
+		t.Log(row)
+	}
+}

dskit/mysql/timeseries.go

@@ -0,0 +1,74 @@
+package mysql
+
+import (
+	"context"
+	"fmt"
+	"strings"
+
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+
+	"gorm.io/gorm"
+)
+
+// Query executes a given SQL query in MySQL and returns the results
+func (m *MySQL) Query(ctx context.Context, query *sqlbase.QueryParam) ([]map[string]interface{}, error) {
+	db, err := m.NewConn(ctx, "")
+	if err != nil {
+		return nil, err
+	}
+
+	err = m.CheckMaxQueryRows(db, ctx, query)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.Query(ctx, db, query)
+}
+
+// QueryTimeseries executes a time series data query using the given parameters
+func (m *MySQL) QueryTimeseries(ctx context.Context, query *sqlbase.QueryParam) ([]types.MetricValues, error) {
+	db, err := m.NewConn(ctx, "")
+	if err != nil {
+		return nil, err
+	}
+
+	err = m.CheckMaxQueryRows(db, ctx, query)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.QueryTimeseries(ctx, db, query)
+}
+
+func (m *MySQL) CheckMaxQueryRows(db *gorm.DB, ctx context.Context, query *sqlbase.QueryParam) error {
+	sql := strings.ReplaceAll(query.Sql, ";", "")
+	checkQuery := &sqlbase.QueryParam{
+		Sql: fmt.Sprintf("SELECT COUNT(*) as count FROM (%s) AS subquery;", sql),
+	}
+
+	res, err := sqlbase.Query(ctx, db, checkQuery)
+	if err != nil {
+		return err
+	}
+
+	if len(res) > 0 {
+		if count, exists := res[0]["count"]; exists {
+			v, err := sqlbase.ParseFloat64Value(count)
+			if err != nil {
+				return err
+			}
+
+			maxQueryRows := m.Shards[0].MaxQueryRows
+			if maxQueryRows == 0 {
+				maxQueryRows = 500
+			}
+
+			if v > float64(maxQueryRows) {
+				return fmt.Errorf("query result rows count %d exceeds the maximum limit %d", int(v), maxQueryRows)
+			}
+		}
+	}
+
+	return nil
+}

dskit/mysql/timeseries_test.go

@@ -0,0 +1,62 @@
+// @Author: Ciusyan 5/11/24
+
+package mysql
+
+import (
+	"context"
+	"testing"
+
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+
+	"github.com/stretchr/testify/require"
+)
+
+func TestQuery(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	param := &sqlbase.QueryParam{
+		Sql: "SELECT * FROM students WHERE id > 10900",
+		Keys: types.Keys{
+			ValueKey:   "",
+			LabelKey:   "",
+			TimeKey:    "",
+			TimeFormat: "",
+		},
+	}
+
+	rows, err := mysql.Query(ctx, param)
+	require.NoError(t, err)
+	for _, row := range rows {
+		t.Log(row)
+	}
+}
+
+func TestQueryTimeseries(t *testing.T) {
+	ctx := context.Background()
+	settings := `{"mysql.addr":"localhost:3306","mysql.user":"root","mysql.password":"root","mysql.maxIdleConns":5,"mysql.maxOpenConns":10,"mysql.connMaxLifetime":30}`
+	mysql, err := NewMySQLWithSettings(ctx, settings)
+	require.NoError(t, err)
+
+	// Prepare a test query parameter
+	param := &sqlbase.QueryParam{
+		Sql: "SELECT id, grade, student_name, a_grade, update_time FROM students WHERE grade > 20000", // Modify SQL query to select specific columns
+		Keys: types.Keys{
+			ValueKey:   "grade a_grade",                 // Set the value key to the column name containing the metric value
+			LabelKey:   "id student_name",               // Set the label key to the column name containing the metric label
+			TimeKey:    "update_time",                   // Set the time key to the column name containing the timestamp
+			TimeFormat: "2006-01-02 15:04:05 +0000 UTC", // Provide the time format according to the timestamp column's format
+		},
+	}
+
+	// Execute the query and retrieve the time series data
+	metricValues, err := mysql.QueryTimeseries(ctx, param)
+	require.NoError(t, err)
+
+	for _, metric := range metricValues {
+		t.Log(metric)
+	}
+}

dskit/pool/pool.go

@@ -0,0 +1,37 @@
+package pool
+
+import (
+	"bytes"
+	"sync"
+	"time"
+
+	gc "github.com/patrickmn/go-cache"
+)
+
+var (
+	PoolClient = new(sync.Map)
+)
+
+var (
+	// default cache instance, do not use this if you want to specify the defaultExpiration
+	DefaultCache = gc.New(time.Hour*24, time.Hour)
+)
+
+var (
+	bytesPool = sync.Pool{
+		New: func() interface{} { return new(bytes.Buffer) },
+	}
+)
+
+func PoolGetBytesBuffer() *bytes.Buffer {
+	buf := bytesPool.Get().(*bytes.Buffer)
+	buf.Reset()
+	return buf
+}
+
+func PoolPutBytesBuffer(buf *bytes.Buffer) {
+	if buf == nil {
+		return
+	}
+	bytesPool.Put(buf)
+}

dskit/postgres/postgres.go

@@ -0,0 +1,207 @@
+// @Author: Ciusyan 5/20/24
+
+package postgres
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/ccfos/nightingale/v6/dskit/pool"
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+
+	_ "github.com/lib/pq" // PostgreSQL driver
+	"github.com/mitchellh/mapstructure"
+	"gorm.io/driver/postgres"
+	"gorm.io/gorm"
+)
+
+type PostgreSQL struct {
+	Shard `json:",inline"  mapstructure:",squash"`
+}
+
+type Shard struct {
+	Addr            string `json:"pgsql.addr" mapstructure:"pgsql.addr"`
+	DB              string `json:"pgsql.db" mapstructure:"pgsql.db"`
+	User            string `json:"pgsql.user" mapstructure:"pgsql.user"`
+	Password        string `json:"pgsql.password" mapstructure:"pgsql.password" `
+	Timeout         int    `json:"pgsql.timeout" mapstructure:"pgsql.timeout"`
+	MaxIdleConns    int    `json:"pgsql.max_idle_conns" mapstructure:"pgsql.max_idle_conns"`
+	MaxOpenConns    int    `json:"pgsql.max_open_conns" mapstructure:"pgsql.max_open_conns"`
+	ConnMaxLifetime int    `json:"pgsql.conn_max_lifetime" mapstructure:"pgsql.conn_max_lifetime"`
+	MaxQueryRows    int    `json:"pgsql.max_query_rows" mapstructure:"pgsql.max_query_rows"`
+}
+
+// NewPostgreSQLWithSettings initializes a new PostgreSQL instance with the given settings
+func NewPostgreSQLWithSettings(ctx context.Context, settings interface{}) (*PostgreSQL, error) {
+	newest := new(PostgreSQL)
+	settingsMap := map[string]interface{}{}
+
+	switch s := settings.(type) {
+	case string:
+		if err := json.Unmarshal([]byte(s), &settingsMap); err != nil {
+			return nil, err
+		}
+	case map[string]interface{}:
+		settingsMap = s
+	case *PostgreSQL:
+		return s, nil
+	case PostgreSQL:
+		return &s, nil
+	case Shard:
+		newest.Shard = s
+		return newest, nil
+	case *Shard:
+		newest.Shard = *s
+		return newest, nil
+	default:
+		return nil, errors.New("unsupported settings type")
+	}
+
+	if err := mapstructure.Decode(settingsMap, newest); err != nil {
+		return nil, err
+	}
+
+	return newest, nil
+}
+
+// NewConn establishes a new connection to PostgreSQL
+func (p *PostgreSQL) NewConn(ctx context.Context, database string) (*gorm.DB, error) {
+	if len(p.DB) == 0 && len(database) == 0 {
+		return nil, errors.New("empty pgsql database") // 兼容阿里实时数仓Holgres, 连接时必须指定db名字
+	}
+
+	if p.Shard.Timeout == 0 {
+		p.Shard.Timeout = 60
+	}
+
+	if p.Shard.MaxIdleConns == 0 {
+		p.Shard.MaxIdleConns = 10
+	}
+
+	if p.Shard.MaxOpenConns == 0 {
+		p.Shard.MaxOpenConns = 100
+	}
+
+	if p.Shard.ConnMaxLifetime == 0 {
+		p.Shard.ConnMaxLifetime = 14400
+	}
+
+	if len(p.Shard.Addr) == 0 {
+		return nil, errors.New("empty fe-node addr")
+	}
+	var keys []string
+	var err error
+	keys = append(keys, p.Shard.Addr)
+
+	keys = append(keys, p.Shard.Password, p.Shard.User)
+	if len(database) > 0 {
+		keys = append(keys, database)
+	}
+	cachedKey := strings.Join(keys, ":")
+	// cache conn with database
+	conn, ok := pool.PoolClient.Load(cachedKey)
+	if ok {
+		return conn.(*gorm.DB), nil
+	}
+
+	var db *gorm.DB
+	defer func() {
+		if db != nil && err == nil {
+			pool.PoolClient.Store(cachedKey, db)
+		}
+	}()
+
+	// Simplified connection logic for PostgreSQL
+	dsn := fmt.Sprintf("postgres://%s:%s@%s/%s?sslmode=disable&TimeZone=Asia/Shanghai", p.Shard.User, p.Shard.Password, p.Shard.Addr, database)
+	db, err = sqlbase.NewDB(
+		ctx,
+		postgres.Open(dsn),
+		p.Shard.MaxIdleConns,
+		p.Shard.MaxOpenConns,
+		time.Duration(p.Shard.ConnMaxLifetime)*time.Second,
+	)
+
+	if err != nil {
+		if db != nil {
+			sqlDB, _ := db.DB()
+			if sqlDB != nil {
+				sqlDB.Close()
+			}
+		}
+		return nil, err
+	}
+
+	return db, nil
+}
+
+// ShowDatabases lists all databases in PostgreSQL
+func (p *PostgreSQL) ShowDatabases(ctx context.Context, searchKeyword string) ([]string, error) {
+	db, err := p.NewConn(ctx, "postgres")
+	if err != nil {
+		return nil, err
+	}
+	sql := fmt.Sprintf("SELECT datname FROM pg_database WHERE datistemplate = false AND datname LIKE %s",
+		"'%"+searchKeyword+"%'")
+	return sqlbase.ShowDatabases(ctx, db, sql)
+}
+
+// ShowTables lists all tables in a given database
+func (p *PostgreSQL) ShowTables(ctx context.Context, searchKeyword string) (map[string][]string, error) {
+	db, err := p.NewConn(ctx, p.DB)
+	if err != nil {
+		return nil, err
+	}
+	sql := fmt.Sprintf("SELECT schemaname, tablename FROM pg_tables WHERE schemaname !='information_schema' and schemaname !='pg_catalog'  and  tablename LIKE %s",
+		"'%"+searchKeyword+"%'")
+	rets, err := sqlbase.ExecQuery(ctx, db, sql)
+	if err != nil {
+		return nil, err
+	}
+	tabs := make(map[string][]string, 3)
+	for _, row := range rets {
+		if val, ok := row["schemaname"].(string); ok {
+			tabs[val] = append(tabs[val], row["tablename"].(string))
+		}
+	}
+	return tabs, nil
+}
+
+// DescTable describes the schema of a specified table in PostgreSQL
+// scheme default: public if not specified
+func (p *PostgreSQL) DescTable(ctx context.Context, scheme, table string) ([]*types.ColumnProperty, error) {
+	db, err := p.NewConn(ctx, p.DB)
+	if err != nil {
+		return nil, err
+	}
+	if scheme == "" {
+		scheme = "public"
+	}
+
+	query := fmt.Sprintf("SELECT column_name, data_type, is_nullable, column_default FROM information_schema.columns WHERE table_name = '%s' AND table_schema = '%s'", table, scheme)
+	return sqlbase.DescTable(ctx, db, query)
+}
+
+// SelectRows selects rows from a specified table in PostgreSQL based on a given query
+func (p *PostgreSQL) SelectRows(ctx context.Context, table, where string) ([]map[string]interface{}, error) {
+	db, err := p.NewConn(ctx, p.DB)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.SelectRows(ctx, db, table, where)
+}
+
+// ExecQuery executes a SQL query in PostgreSQL
+func (p *PostgreSQL) ExecQuery(ctx context.Context, sql string) ([]map[string]interface{}, error) {
+	db, err := p.NewConn(ctx, p.DB)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.ExecQuery(ctx, db, sql)
+}

dskit/postgres/timeseries.go

@@ -0,0 +1,73 @@
+package postgres
+
+import (
+	"context"
+	"fmt"
+	"strings"
+
+	"github.com/ccfos/nightingale/v6/dskit/sqlbase"
+	"github.com/ccfos/nightingale/v6/dskit/types"
+	"gorm.io/gorm"
+)
+
+// Query executes a given SQL query in PostgreSQL and returns the results
+func (p *PostgreSQL) Query(ctx context.Context, query *sqlbase.QueryParam) ([]map[string]interface{}, error) {
+	db, err := p.NewConn(ctx, p.Shard.DB)
+	if err != nil {
+		return nil, err
+	}
+
+	err = p.CheckMaxQueryRows(db, ctx, query)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.Query(ctx, db, query)
+}
+
+// QueryTimeseries executes a time series data query using the given parameters
+func (p *PostgreSQL) QueryTimeseries(ctx context.Context, query *sqlbase.QueryParam) ([]types.MetricValues, error) {
+	db, err := p.NewConn(ctx, p.Shard.DB)
+	if err != nil {
+		return nil, err
+	}
+
+	err = p.CheckMaxQueryRows(db, ctx, query)
+	if err != nil {
+		return nil, err
+	}
+
+	return sqlbase.QueryTimeseries(ctx, db, query, true)
+}
+
+func (p *PostgreSQL) CheckMaxQueryRows(db *gorm.DB, ctx context.Context, query *sqlbase.QueryParam) error {
+	sql := strings.ReplaceAll(query.Sql, ";", "")
+	checkQuery := &sqlbase.QueryParam{
+		Sql: fmt.Sprintf("SELECT COUNT(*) as count FROM (%s) AS subquery;", sql),
+	}
+
+	res, err := sqlbase.Query(ctx, db, checkQuery)
+	if err != nil {
+		return err
+	}
+
+	if len(res) > 0 {
+		if count, exists := res[0]["count"]; exists {
+			v, err := sqlbase.ParseFloat64Value(count)
+			if err != nil {
+				return err
+			}
+
+			maxQueryRows := p.Shard.MaxQueryRows
+			if maxQueryRows == 0 {
+				maxQueryRows = 500
+			}
+
+			if v > float64(maxQueryRows) {
+				return fmt.Errorf("query result rows count %d exceeds the maximum limit %d", int(v), maxQueryRows)
+			}
+		}
+	}
+
+	return nil
+}

dskit/sqlbase/base.go

@@ -9,9 +9,9 @@ import (
 	"strings"
 	"time"
 
-	"github.com/ccfos/nightingale/v6/dskit/types"
-
 	"gorm.io/gorm"
+
+	"github.com/ccfos/nightingale/v6/dskit/types"
 )
 
 // NewDB creates a new Gorm DB instance based on the provided gorm.Dialector and configures the connection pool
@@ -19,7 +19,7 @@ func NewDB(ctx context.Context, dialector gorm.Dialector, maxIdleConns, maxOpenC
 	// Create a new Gorm DB instance
 	db, err := gorm.Open(dialector, &gorm.Config{})
 	if err != nil {
-		return nil, err
+		return db, err
 	}
 
 	// Configure the connection pool
@@ -35,6 +35,17 @@ func NewDB(ctx context.Context, dialector gorm.Dialector, maxIdleConns, maxOpenC
 	return db.WithContext(ctx), sqlDB.Ping()
 }
 
+func CloseDB(db *gorm.DB) error {
+	if db != nil {
+		sqlDb, err := db.DB()
+		if err != nil {
+			return err
+		}
+		return sqlDb.Close()
+	}
+	return nil
+}
+
 // ShowTables retrieves a list of all tables in the specified database
 func ShowTables(ctx context.Context, db *gorm.DB, query string) ([]string, error) {
 	var tables []string
@@ -112,7 +123,7 @@ func DescTable(ctx context.Context, db *gorm.DB, query string) ([]*types.ColumnP
 		}
 
 		// Convert the database-specific type to internal type
-		type2, indexable := convertDBType(db.Dialector.Name(), typ)
+		type2, indexable := ConvertDBType(db.Dialector.Name(), typ)
 		columns = append(columns, &types.ColumnProperty{
 			Field:     field,
 			Type:      typ,
@@ -175,7 +186,7 @@ func SelectRows(ctx context.Context, db *gorm.DB, table, query string) ([]map[st
 }
 
 // convertDBType converts MySQL or PostgreSQL data types to custom internal types and determines if they are indexable
-func convertDBType(dialect, dbType string) (string, bool) {
+func ConvertDBType(dialect, dbType string) (string, bool) {
 	typ := strings.ToLower(dbType)
 
 	// Common type conversions
@@ -190,7 +201,7 @@ func convertDBType(dialect, dbType string) (string, bool) {
 		strings.HasPrefix(typ, "char"), strings.HasPrefix(typ, "tinytext"),
 		strings.HasPrefix(typ, "mediumtext"), strings.HasPrefix(typ, "longtext"),
 		strings.HasPrefix(typ, "character varying"), strings.HasPrefix(typ, "nvarchar"),
-		strings.HasPrefix(typ, "nchar"):
+		strings.HasPrefix(typ, "nchar"), strings.HasPrefix(typ, "bpchar"):
 		return types.LogExtractValueTypeText, true
 
 	case strings.HasPrefix(typ, "float"), strings.HasPrefix(typ, "double"),
@@ -203,7 +214,7 @@ func convertDBType(dialect, dbType string) (string, bool) {
 		strings.HasPrefix(typ, "time"), strings.HasPrefix(typ, "smalldatetime"):
 		return types.LogExtractValueTypeDate, false
 
-	case strings.HasPrefix(typ, "boolean"), strings.HasPrefix(typ, "bit"):
+	case strings.HasPrefix(typ, "boolean"), strings.HasPrefix(typ, "bit"), strings.HasPrefix(typ, "bool"):
 		return types.LogExtractValueTypeBool, false
 	}
 

dskit/sqlbase/timeseries.go

@@ -112,7 +112,8 @@ func FormatMetricValues(keys types.Keys, rows []map[string]interface{}, ignoreDe
 				metricTs[k] = float64(ts.Unix())
 			default:
 				// Default to labels for any unrecognized columns
-				if !ignore {
+				if !ignore && keys.LabelKey == "" {
+					// 只有当 labelKey 为空时，才将剩余的列作为 label
 					labels[k] = fmt.Sprintf("%v", v)
 				}
 			}

go.mod

@@ -27,11 +27,13 @@ require (
 	github.com/jinzhu/copier v0.4.0
 	github.com/json-iterator/go v1.1.12
 	github.com/koding/multiconfig v0.0.0-20171124222453-69c27309b2d7
+	github.com/lib/pq v1.0.0
 	github.com/mailru/easyjson v0.7.7
 	github.com/mattn/go-isatty v0.0.19
 	github.com/mitchellh/mapstructure v1.5.0
 	github.com/mojocn/base64Captcha v1.3.6
 	github.com/olivere/elastic/v7 v7.0.32
+	github.com/patrickmn/go-cache v2.1.0+incompatible
 	github.com/pelletier/go-toml/v2 v2.0.8
 	github.com/pkg/errors v0.9.1
 	github.com/prometheus/client_golang v1.20.5
@@ -120,7 +122,7 @@ require (
 	github.com/go-playground/locales v0.14.1 // indirect
 	github.com/go-playground/universal-translator v0.18.1 // indirect
 	github.com/go-playground/validator/v10 v10.14.0 // indirect
-	github.com/go-sql-driver/mysql v1.6.0 // indirect
+	github.com/go-sql-driver/mysql v1.6.0
 	github.com/goccy/go-json v0.10.2 // indirect
 	github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
 	github.com/grafana/regexp v0.0.0-20221122212121-6b5c0a4cb7fd // indirect

go.sum

@@ -220,6 +220,8 @@ github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+
 github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
 github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
 github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
+github.com/lib/pq v1.0.0 h1:X5PMW56eZitiTeO7tKzZxFCSpbFZJtkMMooicw2us9A=
+github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
 github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
@@ -246,6 +248,8 @@ github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
 github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
 github.com/olivere/elastic/v7 v7.0.32 h1:R7CXvbu8Eq+WlsLgxmKVKPox0oOwAE/2T9Si5BnvK6E=
 github.com/olivere/elastic/v7 v7.0.32/go.mod h1:c7PVmLe3Fxq77PIfY/bZmxY/TAamBhCzZ8xDOE09a9k=
+github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
+github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
 github.com/paulmach/orb v0.11.1 h1:3koVegMC4X/WeiXYz9iswopaTwMem53NzTJuTF20JzU=
 github.com/paulmach/orb v0.11.1/go.mod h1:5mULz1xQfs3bmQm63QEJA6lNGujuRafwA5S/EnuLaLU=
 github.com/paulmach/protoscan v0.2.1/go.mod h1:SpcSwydNLrxUGSDvXvO0P7g7AuhJ7lcKfDlhJCDw2gY=

integrations/Linux/collect/ntp/ntp.toml

@@ -0,0 +1,8 @@
+# # collect interval
+# interval = 15
+
+# # ntp servers
+# ntp_servers = ["ntp.aliyun.com"]
+
+# # response time out seconds
+# timeout = 5

integrations/Linux/markdown/README.md

@@ -53,4 +53,9 @@ nr_alloc_batch = 0
 
 ## arp_package
 
-统计 ARP 包的数量，该插件依赖 cgo，如果需要该插件需要下载 `with-cgo` 的 categraf 发布包。
+统计 ARP 包的数量，该插件依赖 cgo，如果需要该插件需要下载 `with-cgo` 的 categraf 发布包。
+
+
+## ntp
+
+监控机器时间偏移量，只需要给出 ntp 服务端地址，Categraf 就会周期性去请求，对比本机时间，得到偏移量，监控指标是 ntp_offset_ms 顾名思义，单位是毫秒，一般这个值不能超过 1000

memsto/event_processor_cache.go

@@ -20,7 +20,7 @@ type EventProcessorCacheType struct {
 	stats           *Stats
 
 	sync.RWMutex
-	processors map[int64]*models.EventPipeline // key: pipeline id
+	eventPipelines map[int64]*models.EventPipeline // key: pipeline id
 }
 
 func NewEventProcessorCache(ctx *ctx.Context, stats *Stats) *EventProcessorCacheType {
@@ -29,7 +29,7 @@ func NewEventProcessorCache(ctx *ctx.Context, stats *Stats) *EventProcessorCache
 		statLastUpdated: -1,
 		ctx:             ctx,
 		stats:           stats,
-		processors:      make(map[int64]*models.EventPipeline),
+		eventPipelines:  make(map[int64]*models.EventPipeline),
 	}
 	epc.SyncEventProcessors()
 	return epc
@@ -41,7 +41,7 @@ func (epc *EventProcessorCacheType) Reset() {
 
 	epc.statTotal = -1
 	epc.statLastUpdated = -1
-	epc.processors = make(map[int64]*models.EventPipeline)
+	epc.eventPipelines = make(map[int64]*models.EventPipeline)
 }
 
 func (epc *EventProcessorCacheType) StatChanged(total, lastUpdated int64) bool {
@@ -54,7 +54,7 @@ func (epc *EventProcessorCacheType) StatChanged(total, lastUpdated int64) bool {
 
 func (epc *EventProcessorCacheType) Set(m map[int64]*models.EventPipeline, total, lastUpdated int64) {
 	epc.Lock()
-	epc.processors = m
+	epc.eventPipelines = m
 	epc.Unlock()
 
 	// only one goroutine used, so no need lock
@@ -65,17 +65,29 @@ func (epc *EventProcessorCacheType) Set(m map[int64]*models.EventPipeline, total
 func (epc *EventProcessorCacheType) Get(processorId int64) *models.EventPipeline {
 	epc.RLock()
 	defer epc.RUnlock()
-	return epc.processors[processorId]
+	return epc.eventPipelines[processorId]
+}
+
+func (epc *EventProcessorCacheType) GetProcessorsById(processorId int64) []models.Processor {
+	epc.RLock()
+	defer epc.RUnlock()
+
+	eventPipeline, ok := epc.eventPipelines[processorId]
+	if !ok {
+		return []models.Processor{}
+	}
+
+	return eventPipeline.Processors
 }
 
 func (epc *EventProcessorCacheType) GetProcessorIds() []int64 {
 	epc.RLock()
 	defer epc.RUnlock()
 
-	count := len(epc.processors)
+	count := len(epc.eventPipelines)
 	list := make([]int64, 0, count)
-	for processorId := range epc.processors {
-		list = append(list, processorId)
+	for eid := range epc.eventPipelines {
+		list = append(list, eid)
 	}
 
 	return list
@@ -125,7 +137,18 @@ func (epc *EventProcessorCacheType) syncEventProcessors() error {
 
 	m := make(map[int64]*models.EventPipeline)
 	for i := 0; i < len(lst); i++ {
-		m[lst[i].ID] = lst[i]
+		eventPipeline := lst[i]
+		for _, p := range eventPipeline.ProcessorConfigs {
+			processor, err := models.GetProcessorByType(p.Typ, p.Config)
+			if err != nil {
+				logger.Warningf("event_pipeline_id: %d, event:%+v, processor:%+v get processor err: %+v", eventPipeline.ID, eventPipeline, p, err)
+				continue
+			}
+
+			eventPipeline.Processors = append(eventPipeline.Processors, processor)
+		}
+
+		m[lst[i].ID] = eventPipeline
 	}
 
 	epc.Set(m, stat.Total, stat.LastUpdated)

models/alert_aggr_view.go

@@ -16,7 +16,6 @@ type AlertAggrView struct {
 	Id       int64  `json:"id" gorm:"primaryKey"`
 	Name     string `json:"name"`
 	Rule     string `json:"rule"`
-	Format   string `json:"format"`
 	Cate     int    `json:"cate"`
 	CreateAt int64  `json:"create_at"`
 	CreateBy int64  `json:"create_by"`
@@ -38,33 +37,35 @@ func (v *AlertAggrView) Verify() error {
 		return errors.New("rule is blank")
 	}
 
-	var validFields = []string{
-		"cluster",
-		"group_id",
-		"group_name",
-		"rule_id",
-		"rule_name",
-		"severity",
-		"runbook_url",
-		"target_ident",
-		"target_note",
-	}
-
-	arr := strings.Split(v.Rule, "::")
-	for i := 0; i < len(arr); i++ {
-		pair := strings.Split(arr[i], ":")
-		if len(pair) != 2 {
-			return errors.New("rule invalid")
+	if !strings.Contains(v.Rule, "{{") {
+		var validFields = []string{
+			"cluster",
+			"group_id",
+			"group_name",
+			"rule_id",
+			"rule_name",
+			"severity",
+			"runbook_url",
+			"target_ident",
+			"target_note",
 		}
 
-		if !(pair[0] == "field" || pair[0] == "tagkey") {
-			return errors.New("rule invalid")
-		}
+		arr := strings.Split(v.Rule, "::")
+		for i := 0; i < len(arr); i++ {
+			pair := strings.Split(arr[i], ":")
+			if len(pair) != 2 {
+				return errors.New("rule invalid")
+			}
 
-		if pair[0] == "field" {
-			// 只支持有限的field
-			if !slice.ContainsString(validFields, pair[1]) {
-				return fmt.Errorf("unsupported field: %s", pair[1])
+			if !(pair[0] == "field" || pair[0] == "tagkey") {
+				return errors.New("rule invalid")
+			}
+
+			if pair[0] == "field" {
+				// 只支持有限的field
+				if !slice.ContainsString(validFields, pair[1]) {
+					return fmt.Errorf("unsupported field: %s", pair[1])
+				}
 			}
 		}
 	}

models/alert_cur_event.go

@@ -4,6 +4,7 @@ import (
 	"bytes"
 	"encoding/json"
 	"fmt"
+	"net/http"
 	"reflect"
 	"strconv"
 	"strings"
@@ -15,6 +16,7 @@ import (
 	"github.com/ccfos/nightingale/v6/pkg/tplx"
 	"github.com/ccfos/nightingale/v6/pkg/unit"
 
+	"github.com/toolkits/pkg/ginx"
 	"github.com/toolkits/pkg/logger"
 )
 
@@ -74,7 +76,15 @@ type AlertCurEvent struct {
 	RecoverConfig      RecoverConfig       `json:"recover_config" gorm:"-"`
 	RuleHash           string              `json:"rule_hash" gorm:"-"`
 	ExtraInfoMap       []map[string]string `json:"extra_info_map" gorm:"-"`
-	NotifyRuleIDs      []int64             `json:"notify_rule_ids" gorm:"-"`
+	NotifyRuleIds      []int64             `json:"notify_rule_ids" gorm:"serializer:json"`
+
+	NotifyVersion int                `json:"notify_version"  gorm:"-"` // 0: old, 1: new
+	NotifyRules   []*EventNotifyRule `json:"notify_rules" gorm:"-"`
+}
+
+type EventNotifyRule struct {
+	Id   int64  `json:"id"`
+	Name string `json:"name"`
 }
 
 func (e *AlertCurEvent) SetTagsMap() {
@@ -288,19 +298,49 @@ func (e *AlertCurEvent) ParseURL(url string) (string, error) {
 	return body.String(), nil
 }
 
-func (e *AlertCurEvent) GenCardTitle(rules []*AggrRule, format string) (string, error) {
-	if format != "" {
-		tmpl, err := template.New("card_title").Parse(format)
-		if err != nil {
-			return "", err
+func parseAggrRules(rule string) []*AggrRule {
+	aggrRules := strings.Split(rule, "::") // e.g. field:group_name::field:severity::tagkey:ident
+
+	if len(aggrRules) == 0 {
+		ginx.Bomb(http.StatusBadRequest, "rule empty")
+	}
+
+	rules := make([]*AggrRule, len(aggrRules))
+	for i := 0; i < len(aggrRules); i++ {
+		pair := strings.Split(aggrRules[i], ":")
+		if len(pair) != 2 {
+			ginx.Bomb(http.StatusBadRequest, "rule invalid")
 		}
+
+		if !(pair[0] == "field" || pair[0] == "tagkey") {
+			ginx.Bomb(http.StatusBadRequest, "rule invalid")
+		}
+
+		rules[i] = &AggrRule{
+			Type:  pair[0],
+			Value: pair[1],
+		}
+	}
+	return rules
+}
+
+func (e *AlertCurEvent) GenCardTitle(rule string) (string, error) {
+	if strings.Contains(rule, "{{") {
+		// 有 {{ 表示使用的是新的配置方式，使用 go template 进行格式化
+
+		tmpl, err := template.New("card_title").Parse(rule)
+		if err != nil {
+			return fmt.Sprintf("failed to parse card title: %v", err), nil
+		}
+
 		var buf bytes.Buffer
 		if err := tmpl.Execute(&buf, e); err != nil {
-			return "", err
+			return fmt.Sprintf("failed to execute card title: %v", err), nil
 		}
 		return buf.String(), nil
 	}
 
+	rules := parseAggrRules(rule)
 	arr := make([]string, len(rules))
 	for i := 0; i < len(rules); i++ {
 		rule := rules[i]
@@ -314,7 +354,7 @@ func (e *AlertCurEvent) GenCardTitle(rules []*AggrRule, format string) (string,
 		}
 
 		if len(arr[i]) == 0 {
-			arr[i] = "Null"
+			arr[i] = "Others"
 		}
 	}
 	return strings.Join(arr, "::"), nil
@@ -405,6 +445,7 @@ func (e *AlertCurEvent) ToHis(ctx *ctx.Context) *AlertHisEvent {
 		LastEvalTime:     e.LastEvalTime,
 		NotifyCurNumber:  e.NotifyCurNumber,
 		FirstTriggerTime: e.FirstTriggerTime,
+		NotifyRuleIds:    e.NotifyRuleIds,
 	}
 }
 
@@ -420,6 +461,22 @@ func (e *AlertCurEvent) DB2FE() error {
 	if err := json.Unmarshal([]byte(e.RuleConfig), &e.RuleConfigJson); err != nil {
 		return err
 	}
+
+	e.TagsMap = make(map[string]string)
+	for i := 0; i < len(e.TagsJSON); i++ {
+		pair := strings.TrimSpace(e.TagsJSON[i])
+		if pair == "" {
+			continue
+		}
+
+		arr := strings.SplitN(pair, "=", 2)
+		if len(arr) != 2 {
+			continue
+		}
+
+		e.TagsMap[arr[0]] = arr[1]
+	}
+
 	return nil
 }
 
@@ -537,7 +594,7 @@ func (e *AlertCurEvent) FillNotifyGroups(ctx *ctx.Context, cache map[int64]*User
 }
 
 func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime, etime int64,
-	severity int, dsIds []int64, cates []string, ruleId int64, query string, myGroups []int64) (int64, error) {
+	severity []int64, dsIds []int64, cates []string, ruleId int64, query string, eventIds []int64) (int64, error) {
 	session := DB(ctx).Model(&AlertCurEvent{})
 	if stime != 0 && etime != 0 {
 		session = session.Where("trigger_time between ? and ?", stime, etime)
@@ -550,8 +607,8 @@ func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime,
 		session = session.Where("group_id in ?", bgids)
 	}
 
-	if severity >= 0 {
-		session = session.Where("severity = ?", severity)
+	if len(severity) > 0 {
+		session = session.Where("severity in ?", severity)
 	}
 
 	if len(dsIds) > 0 {
@@ -566,10 +623,9 @@ func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime,
 		session = session.Where("rule_id = ?", ruleId)
 	}
 
-	if len(myGroups) > 0 {
-		session = session.Where("group_id in ?", myGroups)
+	if len(eventIds) > 0 {
+		session = session.Where("id in ?", eventIds)
 	}
-
 	if query != "" {
 		arr := strings.Fields(query)
 		for i := 0; i < len(arr); i++ {
@@ -582,7 +638,7 @@ func AlertCurEventTotal(ctx *ctx.Context, prods []string, bgids []int64, stime,
 }
 
 func AlertCurEventsGet(ctx *ctx.Context, prods []string, bgids []int64, stime, etime int64,
-	severity int, dsIds []int64, cates []string, ruleId int64, query string, limit, offset int, myGroups []int64) (
+	severity []int64, dsIds []int64, cates []string, ruleId int64, query string, limit, offset int, eventIds []int64) (
 	[]AlertCurEvent, error) {
 	session := DB(ctx).Model(&AlertCurEvent{})
 
@@ -597,8 +653,8 @@ func AlertCurEventsGet(ctx *ctx.Context, prods []string, bgids []int64, stime, e
 		session = session.Where("group_id in ?", bgids)
 	}
 
-	if severity >= 0 {
-		session = session.Where("severity = ?", severity)
+	if len(severity) > 0 {
+		session = session.Where("severity in ?", severity)
 	}
 
 	if len(dsIds) > 0 {
@@ -612,9 +668,11 @@ func AlertCurEventsGet(ctx *ctx.Context, prods []string, bgids []int64, stime, e
 	if ruleId > 0 {
 		session = session.Where("rule_id = ?", ruleId)
 	}
-	if len(myGroups) > 0 {
-		session = session.Where("group_id in ?", myGroups)
+
+	if len(eventIds) > 0 {
+		session = session.Where("id in ?", eventIds)
 	}
+
 	if query != "" {
 		arr := strings.Fields(query)
 		for i := 0; i < len(arr); i++ {
@@ -975,9 +1033,9 @@ func (e *AlertCurEvent) DeepCopy() *AlertCurEvent {
 		}
 	}
 
-	if e.NotifyRuleIDs != nil {
-		eventCopy.NotifyRuleIDs = make([]int64, len(e.NotifyRuleIDs))
-		copy(eventCopy.NotifyRuleIDs, e.NotifyRuleIDs)
+	if e.NotifyRuleIds != nil {
+		eventCopy.NotifyRuleIds = make([]int64, len(e.NotifyRuleIds))
+		copy(eventCopy.NotifyRuleIds, e.NotifyRuleIds)
 	}
 
 	eventCopy.RuleConfigJson = e.RuleConfigJson

models/alert_his_event.go

@@ -55,6 +55,10 @@ type AlertHisEvent struct {
 	NotifyCurNumber    int               `json:"notify_cur_number"`    // notify: current number
 	FirstTriggerTime   int64             `json:"first_trigger_time"`   // 连续告警的首次告警时间
 	ExtraConfig        interface{}       `json:"extra_config" gorm:"-"`
+	NotifyRuleIds      []int64           `json:"notify_rule_ids" gorm:"serializer:json"`
+
+	NotifyVersion int                `json:"notify_version" gorm:"-"`
+	NotifyRules   []*EventNotifyRule `json:"notify_rules" gorm:"-"`
 }
 
 func (e *AlertHisEvent) TableName() string {
@@ -239,6 +243,15 @@ func AlertHisEventGetById(ctx *ctx.Context, id int64) (*AlertHisEvent, error) {
 	return AlertHisEventGet(ctx, "id=?", id)
 }
 
+func AlertHisEventBatchDelete(ctx *ctx.Context, timestamp int64, severities []int, limit int) (int64, error) {
+	db := DB(ctx).Where("last_eval_time < ?", timestamp)
+	if len(severities) > 0 {
+		db = db.Where("severity IN (?)", severities)
+	}
+	res := db.Limit(limit).Delete(&AlertHisEvent{})
+	return res.RowsAffected, res.Error
+}
+
 func (m *AlertHisEvent) UpdateFieldsMap(ctx *ctx.Context, fields map[string]interface{}) error {
 	return DB(ctx).Model(m).Updates(fields).Error
 }

models/alert_rule.go

@@ -28,6 +28,8 @@ const (
 	PROMETHEUS    = "prometheus"
 	TDENGINE      = "tdengine"
 	ELASTICSEARCH = "elasticsearch"
+	MYSQL         = "mysql"
+	POSTGRESQL    = "pgsql"
 
 	CLICKHOUSE = "ck"
 )
@@ -505,7 +507,7 @@ func (ar *AlertRule) Verify() error {
 	ar.AppendTags = strings.TrimSpace(ar.AppendTags)
 	arr := strings.Fields(ar.AppendTags)
 	for i := 0; i < len(arr); i++ {
-		if len(strings.Split(arr[i], "=")) != 2 {
+		if !strings.Contains(arr[i], "=") {
 			return fmt.Errorf("AppendTags(%s) invalid", arr[i])
 		}
 	}
@@ -838,7 +840,6 @@ func (ar *AlertRule) FillNotifyGroups(ctx *ctx.Context, cache map[int64]*UserGro
 }
 
 func (ar *AlertRule) FE2DB() error {
-
 	if len(ar.EnableStimesJSON) > 0 {
 		ar.EnableStime = strings.Join(ar.EnableStimesJSON, " ")
 		ar.EnableEtime = strings.Join(ar.EnableEtimesJSON, " ")
@@ -866,7 +867,16 @@ func (ar *AlertRule) FE2DB() error {
 	ar.NotifyChannels = strings.Join(ar.NotifyChannelsJSON, " ")
 	ar.NotifyGroups = strings.Join(ar.NotifyGroupsJSON, " ")
 	ar.Callbacks = strings.Join(ar.CallbacksJSON, " ")
-	ar.AppendTags = strings.Join(ar.AppendTagsJSON, " ")
+
+	for i := range ar.AppendTagsJSON {
+		// 后面要把多个标签拼接在一起，所以每个标签里不能有空格
+		ar.AppendTagsJSON[i] = strings.ReplaceAll(ar.AppendTagsJSON[i], " ", "")
+	}
+
+	if len(ar.AppendTagsJSON) > 0 {
+		ar.AppendTags = strings.Join(ar.AppendTagsJSON, " ")
+	}
+
 	algoParamsByte, err := json.Marshal(ar.AlgoParamsJson)
 	if err != nil {
 		return fmt.Errorf("marshal algo_params err:%v", err)
@@ -1175,12 +1185,21 @@ func (ar *AlertRule) IsLokiRule() bool {
 	return ar.Prod == LOKI || ar.Cate == LOKI
 }
 
+func (ar *AlertRule) IsTdengineRule() bool {
+	return ar.Cate == TDENGINE
+}
+
 func (ar *AlertRule) IsHostRule() bool {
 	return ar.Prod == HOST
 }
 
-func (ar *AlertRule) IsTdengineRule() bool {
-	return ar.Cate == TDENGINE
+func (ar *AlertRule) IsInnerRule() bool {
+	return ar.Cate == TDENGINE ||
+		ar.Cate == CLICKHOUSE ||
+		ar.Cate == ELASTICSEARCH ||
+		ar.Prod == LOKI || ar.Cate == LOKI ||
+		ar.Cate == MYSQL ||
+		ar.Cate == POSTGRESQL
 }
 
 func (ar *AlertRule) GetRuleType() string {

models/alert_subscribe.go

@@ -116,6 +116,10 @@ func (s *AlertSubscribe) Verify() error {
 		return errors.New("severities is required")
 	}
 
+	if len(s.NotifyRuleIds) > 0 {
+		return nil
+	}
+
 	if s.UserGroupIds != "" && s.NewChannels == "" {
 		// 如果指定了用户组，那么新告警的通知渠道必须指定，否则容易出现告警规则中没有指定通知渠道，导致订阅通知时，没有通知渠道
 		return errors.New("new_channels is required")
@@ -408,9 +412,9 @@ func (s *AlertSubscribe) ModifyEvent(event *AlertCurEvent) {
 	}
 
 	if len(s.NotifyRuleIds) > 0 {
-		event.NotifyRuleIDs = s.NotifyRuleIds
+		event.NotifyRuleIds = s.NotifyRuleIds
 	} else {
-		event.NotifyRuleIDs = []int64{}
+		event.NotifyRuleIds = []int64{}
 	}
 
 	event.NotifyGroups = s.UserGroupIds

models/event_pipeline.go

@@ -11,22 +11,24 @@ import (
 
 // EventPipeline 事件Pipeline模型
 type EventPipeline struct {
-	ID           int64       `json:"id" gorm:"primaryKey"`
-	Name         string      `json:"name" gorm:"type:varchar(128)"`
-	TeamIds      []int64     `json:"team_ids" gorm:"type:text;serializer:json"`
-	TeamNames    []string    `json:"team_names" gorm:"-"`
-	Description  string      `json:"description" gorm:"type:varchar(255)"`
-	FilterEnable bool        `json:"filter_enable" gorm:"type:tinyint(1)"`
-	LabelFilters []TagFilter `json:"label_filters" gorm:"type:text;serializer:json"`
-	AttrFilters  []TagFilter `json:"attribute_filters" gorm:"type:text;serializer:json"`
-	Processors   []Processor `json:"processors" gorm:"type:text;serializer:json"`
-	CreateAt     int64       `json:"create_at" gorm:"type:bigint"`
-	CreateBy     string      `json:"create_by" gorm:"type:varchar(64)"`
-	UpdateAt     int64       `json:"update_at" gorm:"type:bigint"`
-	UpdateBy     string      `json:"update_by" gorm:"type:varchar(64)"`
+	ID               int64             `json:"id" gorm:"primaryKey"`
+	Name             string            `json:"name" gorm:"type:varchar(128)"`
+	TeamIds          []int64           `json:"team_ids" gorm:"type:text;serializer:json"`
+	TeamNames        []string          `json:"team_names" gorm:"-"`
+	Description      string            `json:"description" gorm:"type:varchar(255)"`
+	FilterEnable     bool              `json:"filter_enable" gorm:"type:bigint"`
+	LabelFilters     []TagFilter       `json:"label_filters" gorm:"type:text;serializer:json"`
+	AttrFilters      []TagFilter       `json:"attribute_filters" gorm:"type:text;serializer:json"`
+	ProcessorConfigs []ProcessorConfig `json:"processors" gorm:"type:text;serializer:json"`
+	CreateAt         int64             `json:"create_at" gorm:"type:bigint"`
+	CreateBy         string            `json:"create_by" gorm:"type:varchar(64)"`
+	UpdateAt         int64             `json:"update_at" gorm:"type:bigint"`
+	UpdateBy         string            `json:"update_by" gorm:"type:varchar(64)"`
+
+	Processors []Processor `json:"-" gorm:"-"`
 }
 
-type Processor struct {
+type ProcessorConfig struct {
 	Typ    string      `json:"typ"`
 	Config interface{} `json:"config"`
 }
@@ -44,22 +46,20 @@ func (e *EventPipeline) Verify() error {
 		return errors.New("team_ids cannot be empty")
 	}
 
-	return nil
-}
-
-func (e *EventPipeline) DB2FE() {
-	if e.TeamIds == nil {
+	if len(e.TeamIds) == 0 {
 		e.TeamIds = make([]int64, 0)
 	}
-	if e.LabelFilters == nil {
+	if len(e.LabelFilters) == 0 {
 		e.LabelFilters = make([]TagFilter, 0)
 	}
-	if e.AttrFilters == nil {
+	if len(e.AttrFilters) == 0 {
 		e.AttrFilters = make([]TagFilter, 0)
 	}
-	if e.Processors == nil {
-		e.Processors = make([]Processor, 0)
+	if len(e.ProcessorConfigs) == 0 {
+		e.ProcessorConfigs = make([]ProcessorConfig, 0)
 	}
+
+	return nil
 }
 
 // CreateEventPipeline 创建事件Pipeline
@@ -74,7 +74,7 @@ func GetEventPipeline(ctx *ctx.Context, id int64) (*EventPipeline, error) {
 	if err != nil {
 		return nil, err
 	}
-	pipeline.DB2FE()
+	pipeline.Verify()
 	return &pipeline, nil
 }
 
@@ -102,13 +102,13 @@ func ListEventPipelines(ctx *ctx.Context) ([]*EventPipeline, error) {
 	}
 
 	var pipelines []*EventPipeline
-	err := DB(ctx).Order("name desc").Find(&pipelines).Error
+	err := DB(ctx).Order("name asc").Find(&pipelines).Error
 	if err != nil {
 		return nil, err
 	}
 
 	for _, p := range pipelines {
-		p.DB2FE()
+		p.Verify()
 	}
 
 	return pipelines, nil

models/event_processor.go

@@ -1,33 +1,25 @@
-package pipeline
+package models
 
 import (
 	"fmt"
 	"strings"
 
-	"github.com/ccfos/nightingale/v6/models"
 	"github.com/ccfos/nightingale/v6/pkg/ctx"
 )
 
-// Processor 是处理器接口，所有处理器类型都需要实现此接口
 type Processor interface {
-	Init(settings interface{}) (Processor, error)          // 初始化配置
-	Process(ctx *ctx.Context, event *models.AlertCurEvent) // 处理告警事件
+	Init(settings interface{}) (Processor, error) // 初始化配置
+	Process(ctx *ctx.Context, event *AlertCurEvent) (*AlertCurEvent, string, error)
+	// 处理器有三种情况：
+	// 1. 处理成功，返回处理后的事件
+	// 2. 处理成功，不需要返回处理后端事件，只返回处理结果，将处理结果放到 string 中，比如 eventdrop callback 处理器
+	// 3. 处理失败，返回错误，将错误放到 error 中
 }
 
-// NewProcessorFn 创建处理器的函数类型
 type NewProcessorFn func(settings interface{}) (Processor, error)
 
-// 处理器注册表，存储各种类型处理器的构造函数
 var processorRegister = map[string]NewProcessorFn{}
 
-// // ProcessorTypes 存储所有支持的处理器类型
-// var Processors map[int64]models.Processor
-
-// func init() {
-// 	Processors = make(map[int64]models.Processor)
-// }
-
-// RegisterProcessor 注册处理器类型
 func RegisterProcessor(typ string, p Processor) {
 	if _, found := processorRegister[typ]; found {
 		return
@@ -35,7 +27,6 @@ func RegisterProcessor(typ string, p Processor) {
 	processorRegister[typ] = p.Init
 }
 
-// GetProcessorByType 根据类型获取处理器实例
 func GetProcessorByType(typ string, settings interface{}) (Processor, error) {
 	typ = strings.TrimSpace(typ)
 	fn, found := processorRegister[typ]

models/migrate/migrate.go

@@ -67,8 +67,8 @@ func MigrateTables(db *gorm.DB) error {
 		&TaskRecord{}, &ChartShare{}, &Target{}, &Configs{}, &Datasource{}, &NotifyTpl{},
 		&Board{}, &BoardBusigroup{}, &Users{}, &SsoConfig{}, &models.BuiltinMetric{},
 		&models.MetricFilter{}, &models.NotificaitonRecord{}, &models.TargetBusiGroup{},
-		&models.UserToken{}, &models.DashAnnotation{}, MessageTemplate{}, NotifyRule{}, NotifyChannelConfig{}, &EsIndexPatternMigrate{}, 
-    &AlertAggrView{}, &models.EventPipeline{}, &models.EmbeddedProduct{}}
+		&models.UserToken{}, &models.DashAnnotation{}, MessageTemplate{}, NotifyRule{}, NotifyChannelConfig{}, &EsIndexPatternMigrate{},
+		&models.EventPipeline{}, &models.EmbeddedProduct{}, &models.SourceToken{}}
 
 	if isPostgres(db) {
 		dts = append(dts, &models.PostgresBuiltinComponent{})
@@ -80,24 +80,20 @@ func MigrateTables(db *gorm.DB) error {
 		dts = append(dts, &imodels.TaskSchedulerHealth{})
 	}
 
-	if !columnHasIndex(db, &AlertHisEvent{}, "original_tags") ||
-		!columnHasIndex(db, &AlertCurEvent{}, "original_tags") {
-		asyncDts := []interface{}{&AlertHisEvent{}, &AlertCurEvent{}}
-
-		go func() {
-			defer func() {
-				if r := recover(); r != nil {
-					logger.Errorf("panic to migrate table: %v", r)
-				}
-			}()
-
-			for _, dt := range asyncDts {
-				if err := db.AutoMigrate(dt); err != nil {
-					logger.Errorf("failed to migrate table %+v err:%v", dt, err)
-				}
+	asyncDts := []interface{}{&AlertHisEvent{}, &AlertCurEvent{}}
+	go func() {
+		defer func() {
+			if r := recover(); r != nil {
+				logger.Errorf("panic to migrate table: %v", r)
 			}
 		}()
-	}
+
+		for _, dt := range asyncDts {
+			if err := db.AutoMigrate(dt); err != nil {
+				logger.Errorf("failed to migrate table %+v err:%v", dt, err)
+			}
+		}
+	}()
 
 	if !db.Migrator().HasTable(&models.BuiltinPayload{}) {
 		dts = append(dts, &models.BuiltinPayload{})
@@ -306,12 +302,14 @@ type TaskRecord struct {
 	EventId int64 `gorm:"column:event_id;bigint(20);not null;default:0;comment:event id;index:idx_event_id"`
 }
 type AlertHisEvent struct {
-	LastEvalTime int64  `gorm:"column:last_eval_time;bigint(20);not null;default:0;comment:for time filter;index:idx_last_eval_time"`
-	OriginalTags string `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
+	LastEvalTime  int64   `gorm:"column:last_eval_time;bigint(20);not null;default:0;comment:for time filter;index:idx_last_eval_time"`
+	OriginalTags  string  `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
+	NotifyRuleIds []int64 `gorm:"column:notify_rule_ids;type:text;serializer:json;comment:notify rule ids"`
 }
 
 type AlertCurEvent struct {
-	OriginalTags string `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
+	OriginalTags  string  `gorm:"column:original_tags;type:text;comment:labels key=val,,k2=v2"`
+	NotifyRuleIds []int64 `gorm:"column:notify_rule_ids;type:text;serializer:json;comment:notify rule ids"`
 }
 
 type Target struct {
@@ -465,11 +463,3 @@ type NotifyChannelConfig struct {
 func (c *NotifyChannelConfig) TableName() string {
 	return "notify_channel"
 }
-
-type AlertAggrView struct {
-	Format string `gorm:"size:2048;not null;default:''"`
-}
-
-func (AlertAggrView) TableName() string {
-	return "alert_aggr_view"
-}

models/notify_channel.go

@@ -157,7 +157,7 @@ func (ncc *NotifyChannelConfig) SendScript(events []*AlertCurEvent, tpl map[stri
 		return "", "", fmt.Errorf("script or path is empty")
 	}
 
-	fpath := ".notify_scriptt"
+	fpath := ".notify_script_" + strconv.FormatInt(ncc.ID, 10)
 	if config.Path != "" {
 		fpath = config.Path
 	} else {
@@ -388,7 +388,10 @@ func (ncc *NotifyChannelConfig) SendFlashDuty(events []*AlertCurEvent, flashDuty
 
 	// 设置 URL 参数
 	query := req.URL.Query()
-	query.Add("channel_id", strconv.FormatInt(flashDutyChannelID, 10))
+	if flashDutyChannelID != 0 {
+		// 如果 flashduty 有配置协作空间(channel_id)，则传入 channel_id 参数
+		query.Add("channel_id", strconv.FormatInt(flashDutyChannelID, 10))
+	}
 	req.URL.RawQuery = query.Encode()
 	req.Header.Add("Content-Type", "application/json")
 

models/notify_rule.go

@@ -216,7 +216,7 @@ func NotifyRulesGet(ctx *ctx.Context, where string, args ...interface{}) ([]*Not
 	if where != "" && len(args) > 0 {
 		session = session.Where(where, args...)
 	}
-	err := session.Find(&lst).Error
+	err := session.Order("name asc").Find(&lst).Error
 	if err != nil {
 		return nil, err
 	}

models/prom_alert_rule.go

@@ -2,6 +2,7 @@ package models
 
 import (
 	"fmt"
+	"strings"
 	"time"
 
 	"github.com/toolkits/pkg/logger"
@@ -40,7 +41,7 @@ func ConvertAlert(rule PromRule, interval string, datasouceQueries []DatasourceQ
 	if len(rule.Labels) > 0 {
 		for k, v := range rule.Labels {
 			if k != "severity" {
-				appendTags = append(appendTags, fmt.Sprintf("%s=%s", k, v))
+				appendTags = append(appendTags, fmt.Sprintf("%s=%s", strings.ReplaceAll(k, " ", ""), strings.ReplaceAll(v, " ", "")))
 			} else {
 				switch v {
 				case "critical":

models/source_token.go

@@ -0,0 +1,48 @@
+package models
+
+import (
+	"time"
+
+	"github.com/ccfos/nightingale/v6/pkg/ctx"
+)
+
+type SourceToken struct {
+	Id         int64  `json:"id" gorm:"primaryKey"`
+	SourceType string `json:"source_type" gorm:"column:source_type;type:varchar(64);not null;default:''"`
+	SourceId   string `json:"source_id" gorm:"column:source_id;type:varchar(255);not null;default:''"`
+	Token      string `json:"token" gorm:"column:token;type:varchar(255);not null;default:''"`
+	ExpireAt   int64  `json:"expire_at" gorm:"type:bigint;not null;default:0"`
+	CreateAt   int64  `json:"create_at" gorm:"type:bigint;not null;default:0"`
+	CreateBy   string `json:"create_by" gorm:"type:varchar(64);not null;default:''"`
+}
+
+func (SourceToken) TableName() string {
+	return "source_token"
+}
+
+func (st *SourceToken) Add(ctx *ctx.Context) error {
+	return Insert(ctx, st)
+}
+
+// GetSourceTokenBySource 根据源类型和源ID获取源令牌
+func GetSourceTokenBySource(ctx *ctx.Context, sourceType, sourceId, token string) (*SourceToken, error) {
+	var st SourceToken
+	err := DB(ctx).Where("source_type = ? AND source_id = ? AND token = ?", sourceType, sourceId, token).First(&st).Error
+	if err != nil {
+		return nil, err
+	}
+	return &st, nil
+}
+
+func (st *SourceToken) IsExpired() bool {
+	if st.ExpireAt == 0 {
+		return false // 0 表示永不过期
+	}
+	return time.Now().Unix() > st.ExpireAt
+}
+
+func CleanupExpiredTokens(ctx *ctx.Context) (int64, error) {
+	now := time.Now().Unix()
+	result := DB(ctx).Where("expire_at > 0 AND expire_at < ?", now).Delete(&SourceToken{})
+	return result.RowsAffected, result.Error
+}

pkg/i18nx/var.go

@@ -137,6 +137,9 @@ var I18N = `{
 
     "Organization": "人员组织",
     "User - View": "用户 - 查看",
+    "User - Add": "用户 - 新增",
+    "User - Modify": "用户 - 修改",
+    "User - Delete": "用户 - 删除",
     "Team - View": "团队 - 查看",
     "Team - Add": "团队 - 新增",
     "Team - Modify": "团队 - 修改",
@@ -146,6 +149,9 @@ var I18N = `{
     "Business Group - Modify": "业务组 - 修改",
     "Business Group - Delete": "业务组 - 删除",
     "Role - View": "角色 - 查看",
+    "Role - Add": "角色 - 新增",
+    "Role - Modify": "角色 - 修改",
+    "Role - Delete": "角色 - 删除",
 
     "System Settings": "系统配置",
     "View Site Settings": "查看站点设置",
@@ -295,6 +301,9 @@ var I18N = `{
 
     "Organization": "人員組織",
     "User - View": "用戶 - 查看",
+    "User - Add": "用戶 - 新增",
+    "User - Modify": "用戶 - 修改",
+    "User - Delete": "用戶 - 刪除",
     "Team - View": "團隊 - 查看",
     "Team - Add": "團隊 - 新增",
     "Team - Modify": "團隊 - 修改",
@@ -304,6 +313,9 @@ var I18N = `{
     "Business Group - Modify": "業務組 - 修改",
     "Business Group - Delete": "業務組 - 删除",
     "Role - View": "角色 - 查看",
+    "Role - Add": "角色 - 新增",
+    "Role - Modify": "角色 - 修改",
+    "Role - Delete": "角色 - 删除",
 
     "System Settings": "系統配置",
     "View Site Settings": "查看站點設置",
@@ -450,6 +462,9 @@ var I18N = `{
 
     "Organization": "組織",
     "User - View": "ユーザー - 閲覧",
+    "User - Add": "ユーザー - 追加",
+    "User - Modify": "ユーザー - 修正",
+    "User - Delete": "ユーザー - 削除",
     "Team - View": "チーム - 閲覧",
     "Team - Add": "チーム - 追加",
     "Team - Modify": "チーム - 修正",
@@ -459,6 +474,9 @@ var I18N = `{
     "Business Group - Modify": "業務グループ - 修正",
     "Business Group - Delete": "業務グループ - 削除",
     "Role - View": "役割 - 閲覧",
+    "Role - Add": "役割 - 追加",
+    "Role - Modify": "役割 - 修正",
+    "Role - Delete": "役割 - 削除",
 
     "System Settings": "システム設定",
     "View Site Settings": "サイト設定の表示",
@@ -605,6 +623,9 @@ var I18N = `{
 
     "Organization": "Организация",
     "User - View": "Пользователи - Просмотр",
+    "User - Add": "Пользователи - Добавить",
+    "User - Modify": "Пользователи - Изменить",
+    "User - Delete": "Пользователи - Удалить",
     "Team - View": "Команды - Просмотр",
     "Team - Add": "Команды - Добавить",
     "Team - Modify": "Команды - Изменить",
@@ -614,6 +635,9 @@ var I18N = `{
     "Business Group - Modify": "Бизнес-группы - Изменить",
     "Business Group - Delete": "Бизнес-группы - Удалить",
     "Role - View": "Роли - Просмотр",
+    "Role - Add": "Роли - Добавить",
+    "Role - Modify": "Роли - Изменить",
+    "Role - Delete": "Роли - Удалить",
 
     "System Settings": "Настройки системы",
     "View Site Settings": "Просмотр настроек сайта",