Compare commits

..

28 Commits

Author SHA1 Message Date
ning
70cb2afc7c code refactor 2024-06-14 11:52:56 +08:00
ning
a814bb39b8 code refactor 2024-06-14 11:39:21 +08:00
ning
ece1e4305d feat: oidc support default team 2024-06-14 11:28:49 +08:00
Yening Qin
0a0049c6fb feat: callback support send event to im and remove alert subrule callback resend (#1992)
* feat: Callback operation adds IM connection function (#1984)

* refactor: change alert sub callback

---------

Co-authored-by: Yang Zhiyan <101268302+Yziyan@users.noreply.github.com>
2024-06-14 00:38:02 +08:00
Ulric Qin
1b56ebe62e Merge branch 'main' of github.com:ccfos/nightingale 2024-06-13 15:54:39 +08:00
Ulric Qin
a5e92b95b0 add link in github issue template 2024-06-13 15:54:24 +08:00
ulricqin
8e9d06d43e Update README.md 2024-06-13 15:03:04 +08:00
Ulric Qin
ab289de785 update github issue template 2024-06-13 12:32:47 +08:00
Ulric Qin
8667b7743a Merge branch 'main' of github.com:ccfos/nightingale 2024-06-13 12:21:10 +08:00
Ulric Qin
45b9436f69 update github issue template 2024-06-13 12:20:56 +08:00
ning
3d03bcf329 docs: add perm point 2024-06-11 19:10:04 +08:00
ning
1851601889 refactor: get usergroups service api 2024-06-07 20:07:13 +08:00
ning
fa9745decf refactor: update event api 2024-06-06 17:55:08 +08:00
ning
6f007deeaa refactor: change get list api 2024-06-06 16:42:17 +08:00
ning
8fad705065 fix: edge alert use ibex 2024-06-06 16:07:18 +08:00
ning
675076779e refactor: ibex migrate add charset 2024-06-06 12:13:22 +08:00
710leo
b9e78eee22 docs: change action 2024-06-05 22:26:26 +08:00
710leo
2219584abb docs: change action 2024-06-05 22:16:42 +08:00
710leo
ebe31fd6bc docs: change action 2024-06-05 22:12:12 +08:00
nīng
95ca69e170 docs: change action 2024-06-05 22:04:56 +08:00
nīng
ef1b5d8d16 docs: change action 2024-06-05 21:51:23 +08:00
ning
5b375cf037 docs: change action 2024-06-05 19:42:39 +08:00
ning
108b729cae Merge branch 'main' of github.com:ccfos/nightingale 2024-06-05 18:11:44 +08:00
ning
a385972fa9 refactor: add some i18n 2024-06-05 18:11:31 +08:00
yuweizzz
98a0a9d94c feat: support sqlite (#1978)
* demo sqlite
2024-06-05 17:28:56 +08:00
ning
c79eec648d fix: n9e-edge ibex 2024-06-05 17:12:52 +08:00
Yening Qin
603eadd1f2 feat: alert event support recovery value (#1982)
* feature: the alert response event supports query recovery values (#1975)

* refactor: rule note use

---------

Co-authored-by: Yang Zhiyan <101268302+Yziyan@users.noreply.github.com>
2024-06-05 17:01:31 +08:00
Yening Qin
61a2f552be refactor: integration init (#1981) 2024-06-05 15:14:01 +08:00
45 changed files with 2685 additions and 387 deletions

View File

@@ -1,67 +0,0 @@
name: Bug Report
description: Report a bug encountered while running Nightingale
labels: ["kind/bug"]
body:
- type: markdown
attributes:
value: |
Thanks for taking time to fill out this bug report!
The more detailed the form is filled in, the easier the problem will be solved.
- type: textarea
id: config
attributes:
label: Your config.toml
description: Place config in the toml code section. This will be automatically formatted into toml, so no need for backticks.
render: toml
validations:
required: true
- type: textarea
id: logs
attributes:
label: Relevant logs
description: categraf | telegraf | n9e | prometheus | chrome request/response ...
render: text
validations:
required: true
- type: input
id: system-info
attributes:
label: System info
description: Include nightingale version, operating system, and other relevant details
placeholder: ex. n9e 5.9.2, n9e-fe 5.5.0, categraf 0.1.0, Ubuntu 20.04, Docker 20.10.8
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
description: Describe the steps to reproduce the bug.
value: |
1.
2.
3.
...
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: Expected behavior
description: Describe what you expected to happen when you performed the above steps.
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: Actual behavior
description: Describe what actually happened when you performed the above steps.
validations:
required: true
- type: textarea
id: additional-info
attributes:
label: Additional info
description: Include gist of relevant config, logs, etc.
validations:
required: false

33
.github/ISSUE_TEMPLATE/question.yml vendored Normal file
View File

@@ -0,0 +1,33 @@
name: Bug Report & Usage Question
description: Reporting a bug or asking a question about how to use Nightingale
labels: []
body:
- type: markdown
attributes:
value: |
The more detailed the form is filled in, the easier the problem will be solved.
提供的信息越详细,问题解决的可能性就越大。另外, 提问之前请先搜索历史 issue (包括 close 的), 以免重复提问。
- type: textarea
id: question
attributes:
label: Question and Steps to reproduce
description: Describe your question and steps to reproduce the bug. 描述问题以及复现步骤
validations:
required: true
- type: textarea
id: logs
attributes:
label: Relevant logs and configurations
description: Relevant logs and configurations. 报错日志([查看方法](https://flashcat.cloud/docs/content/flashcat-monitor/nightingale-v6/faq/how-to-check-logs/))以及各个相关组件的配置信息
render: text
validations:
required: true
- type: textarea
id: system-info
attributes:
label: Version
description: Include nightingale version, operating system, and other relevant details. 请告知夜莺的版本、操作系统的版本、CPU架构等信息
validations:
required: true

View File

@@ -26,7 +26,8 @@ jobs:
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v3
with:
version: latest
distribution: goreleaser
version: '~> v1'
args: release --rm-dist
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -36,7 +36,7 @@
## 快速开始
- 👉[文档中心](https://flashcat.cloud/docs/) | [下载中心](https://flashcat.cloud/download/nightingale/)
- ❤️[报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=kind%2Fbug&projects=&template=bug_report.yml)
- ❤️[报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml)
- ℹ️为了提供更快速的访问体验,上述文档和下载站点托管于 [FlashcatCloud](https://flashcat.cloud)
## 功能特点
@@ -82,7 +82,7 @@
- [ ] 仪表盘:支持内嵌 Grafana
- [ ] 告警规则:通知时支持配置过滤标签,避免告警事件中一堆不重要的标签
- [ ] 告警规则:支持配置恢复时的 Promql告警恢复通知也可以带上恢复时的值了
- [x] 告警规则:支持配置恢复时的 Promql告警恢复通知也可以带上恢复时的值了
- [ ] 机器管理自定义标签拆分管理agent 自动上报的标签和用户在页面自定义的标签分开管理,对于 agent 自动上报的标签,以 agent 为准,直接覆盖服务端 DB 中的数据
- [ ] 机器管理:机器支持角色字段,即无头标签,用于描述混部场景
- [ ] 机器管理:把业务组的 busigroup 标签迁移到机器的属性里,让机器支持挂到多个业务组

View File

@@ -106,7 +106,7 @@ func Start(alertc aconf.Alert, pushgwc pconf.Pushgw, syncStats *memsto.Stats, al
busiGroupCache, alertMuteCache, datasourceCache, promClients, tdendgineClients, naming, ctx, alertStats)
dp := dispatch.NewDispatch(alertRuleCache, userCache, userGroupCache, alertSubscribeCache, targetCache, notifyConfigCache, taskTplsCache, alertc.Alerting, ctx, alertStats)
consumer := dispatch.NewConsumer(alertc.Alerting, ctx, dp)
consumer := dispatch.NewConsumer(alertc.Alerting, ctx, dp, promClients)
go dp.ReloadTpls()
go consumer.LoopConsume()

View File

@@ -1,14 +1,19 @@
package dispatch
import (
"encoding/json"
"fmt"
"strings"
"time"
"github.com/ccfos/nightingale/v6/alert/aconf"
"github.com/ccfos/nightingale/v6/alert/common"
"github.com/ccfos/nightingale/v6/alert/queue"
"github.com/ccfos/nightingale/v6/models"
"github.com/ccfos/nightingale/v6/pkg/ctx"
"github.com/ccfos/nightingale/v6/pkg/poster"
promsdk "github.com/ccfos/nightingale/v6/pkg/prom"
"github.com/ccfos/nightingale/v6/prom"
"github.com/toolkits/pkg/concurrent/semaphore"
"github.com/toolkits/pkg/logger"
@@ -18,15 +23,17 @@ type Consumer struct {
alerting aconf.Alerting
ctx *ctx.Context
dispatch *Dispatch
dispatch *Dispatch
promClients *prom.PromClientMap
}
// 创建一个 Consumer 实例
func NewConsumer(alerting aconf.Alerting, ctx *ctx.Context, dispatch *Dispatch) *Consumer {
func NewConsumer(alerting aconf.Alerting, ctx *ctx.Context, dispatch *Dispatch, promClients *prom.PromClientMap) *Consumer {
return &Consumer{
alerting: alerting,
ctx: ctx,
dispatch: dispatch,
alerting: alerting,
ctx: ctx,
dispatch: dispatch,
promClients: promClients,
}
}
@@ -73,17 +80,19 @@ func (e *Consumer) consumeOne(event *models.AlertCurEvent) {
event.RuleName = fmt.Sprintf("failed to parse rule name: %v", err)
}
if err := event.ParseRule("rule_note"); err != nil {
logger.Warningf("ruleid:%d failed to parse rule note: %v", event.RuleId, err)
event.RuleNote = fmt.Sprintf("failed to parse rule note: %v", err)
}
if err := event.ParseRule("annotations"); err != nil {
logger.Warningf("ruleid:%d failed to parse annotations: %v", event.RuleId, err)
event.Annotations = fmt.Sprintf("failed to parse annotations: %v", err)
event.AnnotationsJSON["error"] = event.Annotations
}
e.queryRecoveryVal(event)
if err := event.ParseRule("rule_note"); err != nil {
logger.Warningf("ruleid:%d failed to parse rule note: %v", event.RuleId, err)
event.RuleNote = fmt.Sprintf("failed to parse rule note: %v", err)
}
e.persist(event)
if event.IsRecovered && event.NotifyRecovered == 0 {
@@ -115,3 +124,68 @@ func (e *Consumer) persist(event *models.AlertCurEvent) {
e.dispatch.Astats.CounterRuleEvalErrorTotal.WithLabelValues(fmt.Sprintf("%v", event.DatasourceId), "persist_event").Inc()
}
}
func (e *Consumer) queryRecoveryVal(event *models.AlertCurEvent) {
if !event.IsRecovered {
return
}
// If the event is a recovery event, execute the recovery_promql query
promql, ok := event.AnnotationsJSON["recovery_promql"]
if !ok {
return
}
promql = strings.TrimSpace(promql)
if promql == "" {
logger.Warningf("rule_eval:%s promql is blank", getKey(event))
return
}
if e.promClients.IsNil(event.DatasourceId) {
logger.Warningf("rule_eval:%s error reader client is nil", getKey(event))
return
}
readerClient := e.promClients.GetCli(event.DatasourceId)
var warnings promsdk.Warnings
value, warnings, err := readerClient.Query(e.ctx.Ctx, promql, time.Now())
if err != nil {
logger.Errorf("rule_eval:%s promql:%s, error:%v", getKey(event), promql, err)
event.AnnotationsJSON["recovery_promql_error"] = fmt.Sprintf("promql:%s error:%v", promql, err)
b, err := json.Marshal(event.AnnotationsJSON)
if err != nil {
event.AnnotationsJSON = make(map[string]string)
event.AnnotationsJSON["error"] = fmt.Sprintf("failed to parse annotations: %v", err)
} else {
event.Annotations = string(b)
}
return
}
if len(warnings) > 0 {
logger.Errorf("rule_eval:%s promql:%s, warnings:%v", getKey(event), promql, warnings)
}
anomalyPoints := common.ConvertAnomalyPoints(value)
if len(anomalyPoints) == 0 {
logger.Warningf("rule_eval:%s promql:%s, result is empty", getKey(event), promql)
event.AnnotationsJSON["recovery_promql_error"] = fmt.Sprintf("promql:%s error:%s", promql, "result is empty")
} else {
event.AnnotationsJSON["recovery_value"] = fmt.Sprintf("%v", anomalyPoints[0].Value)
}
b, err := json.Marshal(event.AnnotationsJSON)
if err != nil {
event.AnnotationsJSON = make(map[string]string)
event.AnnotationsJSON["error"] = fmt.Sprintf("failed to parse annotations: %v", err)
} else {
event.Annotations = string(b)
}
}
func getKey(event *models.AlertCurEvent) string {
return common.RuleKey(event.DatasourceId, event.RuleId)
}

View File

@@ -4,7 +4,9 @@ import (
"bytes"
"encoding/json"
"html/template"
"net/url"
"strconv"
"strings"
"sync"
"time"
@@ -31,6 +33,7 @@ type Dispatch struct {
alerting aconf.Alerting
Senders map[string]sender.Sender
CallBacks map[string]sender.CallBacker
tpls map[string]*template.Template
ExtraSenders map[string]sender.Sender
BeforeSenderHook func(*models.AlertCurEvent) bool
@@ -99,6 +102,17 @@ func (e *Dispatch) relaodTpls() error {
models.FeishuCard: sender.NewSender(models.FeishuCard, tmpTpls),
}
// domain -> Callback()
callbacks := map[string]sender.CallBacker{
models.DingtalkDomain: sender.NewCallBacker(models.DingtalkDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
models.WecomDomain: sender.NewCallBacker(models.WecomDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
models.FeishuDomain: sender.NewCallBacker(models.FeishuDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
models.TelegramDomain: sender.NewCallBacker(models.TelegramDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
models.FeishuCardDomain: sender.NewCallBacker(models.FeishuCardDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
models.IbexDomain: sender.NewCallBacker(models.IbexDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
models.DefaultDomain: sender.NewCallBacker(models.DefaultDomain, e.targetCache, e.userCache, e.taskTplsCache, tmpTpls),
}
e.RwLock.RLock()
for channelName, extraSender := range e.ExtraSenders {
senders[channelName] = extraSender
@@ -108,6 +122,7 @@ func (e *Dispatch) relaodTpls() error {
e.RwLock.Lock()
e.tpls = tmpTpls
e.Senders = senders
e.CallBacks = callbacks
e.RwLock.Unlock()
return nil
}
@@ -243,7 +258,7 @@ func (e *Dispatch) Send(rule *models.AlertRule, event *models.AlertCurEvent, not
}
// handle event callbacks
sender.SendCallbacks(e.ctx, notifyTarget.ToCallbackList(), event, e.targetCache, e.userCache, e.taskTplsCache, e.Astats)
e.SendCallbacks(rule, notifyTarget, event)
// handle global webhooks
sender.SendWebhooks(notifyTarget.ToWebhookList(), event, e.Astats)
@@ -252,6 +267,47 @@ func (e *Dispatch) Send(rule *models.AlertRule, event *models.AlertCurEvent, not
go sender.MayPluginNotify(e.genNoticeBytes(event), e.notifyConfigCache.GetNotifyScript(), e.Astats)
}
func (e *Dispatch) SendCallbacks(rule *models.AlertRule, notifyTarget *NotifyTarget, event *models.AlertCurEvent) {
uids := notifyTarget.ToUidList()
urls := notifyTarget.ToCallbackList()
for _, urlStr := range urls {
if len(urlStr) == 0 {
continue
}
cbCtx := sender.BuildCallBackContext(e.ctx, urlStr, rule, []*models.AlertCurEvent{event}, uids, e.userCache, e.Astats)
if strings.HasPrefix(urlStr, "${ibex}") {
e.CallBacks[models.IbexDomain].CallBack(cbCtx)
continue
}
if !(strings.HasPrefix(urlStr, "http://") || strings.HasPrefix(urlStr, "https://")) {
cbCtx.CallBackURL = "http://" + urlStr
}
parsedURL, err := url.Parse(urlStr)
if err != nil {
logger.Errorf("SendCallbacks: failed to url.Parse(urlStr=%s): %v", urlStr, err)
continue
}
// process feishu card
if parsedURL.Host == models.FeishuDomain && parsedURL.Query().Get("card") == "1" {
e.CallBacks[models.FeishuCardDomain].CallBack(cbCtx)
continue
}
callBacker, ok := e.CallBacks[parsedURL.Host]
if ok {
callBacker.CallBack(cbCtx)
} else {
e.CallBacks[models.DefaultDomain].CallBack(cbCtx)
}
}
}
type Notice struct {
Event *models.AlertCurEvent `json:"event"`
Tpls map[string]string `json:"tpls"`

View File

@@ -84,6 +84,14 @@ func (s *NotifyTarget) ToWebhookList() []*models.Webhook {
return webhooks
}
func (s *NotifyTarget) ToUidList() []int64 {
uids := make([]int64, len(s.userMap))
for uid, _ := range s.userMap {
uids = append(uids, uid)
}
return uids
}
// Dispatch 抽象由告警事件到信息接收者的路由策略
// rule: 告警规则
// event: 告警事件

View File

@@ -1,9 +1,8 @@
package sender
import (
"encoding/json"
"fmt"
"strconv"
"html/template"
"net/url"
"strings"
"time"
@@ -13,37 +12,122 @@ import (
"github.com/ccfos/nightingale/v6/pkg/ctx"
"github.com/ccfos/nightingale/v6/pkg/poster"
imodels "github.com/flashcatcloud/ibex/src/models"
"github.com/flashcatcloud/ibex/src/storage"
"github.com/toolkits/pkg/logger"
)
func SendCallbacks(ctx *ctx.Context, urls []string, event *models.AlertCurEvent, targetCache *memsto.TargetCacheType, userCache *memsto.UserCacheType,
taskTplCache *memsto.TaskTplCache, stats *astats.Stats) {
for _, url := range urls {
if url == "" {
continue
}
type (
// CallBacker 进行回调的接口
CallBacker interface {
CallBack(ctx CallBackContext)
}
if strings.HasPrefix(url, "${ibex}") {
if !event.IsRecovered {
handleIbex(ctx, url, event, targetCache, userCache, taskTplCache)
}
continue
}
// CallBackContext 回调时所需的上下文
CallBackContext struct {
Ctx *ctx.Context
CallBackURL string
Users []*models.User
Rule *models.AlertRule
Events []*models.AlertCurEvent
Stats *astats.Stats
}
if !(strings.HasPrefix(url, "http://") || strings.HasPrefix(url, "https://")) {
url = "http://" + url
}
DefaultCallBacker struct{}
)
stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
resp, code, err := poster.PostJSON(url, 5*time.Second, event, 3)
if err != nil {
logger.Errorf("event_callback_fail(rule_id=%d url=%s), resp: %s, err: %v, code: %d", event.RuleId, url, string(resp), err, code)
stats.AlertNotifyErrorTotal.WithLabelValues("rule_callback").Inc()
} else {
logger.Infof("event_callback_succ(rule_id=%d url=%s), resp: %s, code: %d", event.RuleId, url, string(resp), code)
func BuildCallBackContext(ctx *ctx.Context, callBackURL string, rule *models.AlertRule, events []*models.AlertCurEvent,
uids []int64, userCache *memsto.UserCacheType, stats *astats.Stats) CallBackContext {
users := userCache.GetByUserIds(uids)
return CallBackContext{
Ctx: ctx,
CallBackURL: callBackURL,
Rule: rule,
Events: events,
Users: users,
Stats: stats,
}
}
func ExtractAtsParams(rawURL string) []string {
ans := make([]string, 0, 1)
parsedURL, err := url.Parse(rawURL)
if err != nil {
logger.Errorf("ExtractAtsParams(url=%s), err: %v", rawURL, err)
return ans
}
queryParams := parsedURL.Query()
atParam := queryParams.Get("ats")
if atParam == "" {
return ans
}
// Split the atParam by comma and return the result as a slice
return strings.Split(atParam, ",")
}
func NewCallBacker(
key string,
targetCache *memsto.TargetCacheType,
userCache *memsto.UserCacheType,
taskTplCache *memsto.TaskTplCache,
tpls map[string]*template.Template,
) CallBacker {
switch key {
case models.IbexDomain: // Distribute to Ibex
return &IbexCallBacker{
targetCache: targetCache,
userCache: userCache,
taskTplCache: taskTplCache,
}
case models.DefaultDomain: // default callback
return &DefaultCallBacker{}
case models.DingtalkDomain:
return &DingtalkSender{tpl: tpls[models.Dingtalk]}
case models.WecomDomain:
return &WecomSender{tpl: tpls[models.Wecom]}
case models.FeishuDomain:
return &FeishuSender{tpl: tpls[models.Feishu]}
case models.FeishuCardDomain:
return &FeishuCardSender{tpl: tpls[models.FeishuCard]}
//case models.Mm:
// return &MmSender{tpl: tpls[models.Mm]}
case models.TelegramDomain:
return &TelegramSender{tpl: tpls[models.Telegram]}
}
return nil
}
func (c *DefaultCallBacker) CallBack(ctx CallBackContext) {
if len(ctx.CallBackURL) == 0 || len(ctx.Events) == 0 {
return
}
event := ctx.Events[0]
ctx.Stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
resp, code, err := poster.PostJSON(ctx.CallBackURL, 5*time.Second, event, 3)
if err != nil {
logger.Errorf("event_callback_fail(rule_id=%d url=%s), resp: %s, err: %v, code: %d",
event.RuleId, ctx.CallBackURL, string(resp), err, code)
ctx.Stats.AlertNotifyErrorTotal.WithLabelValues("rule_callback").Inc()
} else {
logger.Infof("event_callback_succ(rule_id=%d url=%s), resp: %s, code: %d",
event.RuleId, ctx.CallBackURL, string(resp), code)
}
}
func doSend(url string, body interface{}, channel string, stats *astats.Stats) {
stats.AlertNotifyTotal.WithLabelValues(channel).Inc()
res, code, err := poster.PostJSON(url, time.Second*5, body, 3)
if err != nil {
logger.Errorf("%s_sender: result=fail url=%s code=%d error=%v req:%v response=%s", channel, url, code, err, body, string(res))
stats.AlertNotifyErrorTotal.WithLabelValues(channel).Inc()
} else {
logger.Infof("%s_sender: result=succ url=%s code=%d req:%v response=%s", channel, url, code, body, string(res))
}
}
@@ -51,226 +135,3 @@ type TaskCreateReply struct {
Err string `json:"err"`
Dat int64 `json:"dat"` // task.id
}
func handleIbex(ctx *ctx.Context, url string, event *models.AlertCurEvent, targetCache *memsto.TargetCacheType, userCache *memsto.UserCacheType,
taskTplCache *memsto.TaskTplCache) {
if imodels.DB() == nil {
logger.Warning("event_callback_ibex: db is nil")
return
}
arr := strings.Split(url, "/")
var idstr string
var host string
if len(arr) > 1 {
idstr = arr[1]
}
if len(arr) > 2 {
host = arr[2]
}
id, err := strconv.ParseInt(idstr, 10, 64)
if err != nil {
logger.Errorf("event_callback_ibex: failed to parse url: %s", url)
return
}
if host == "" {
// 用户在callback url中没有传入host就从event中解析
host = event.TargetIdent
}
if host == "" {
logger.Error("event_callback_ibex: failed to get host")
return
}
tpl := taskTplCache.Get(id)
if tpl == nil {
logger.Errorf("event_callback_ibex: no such tpl(%d)", id)
return
}
// check perm
// tpl.GroupId - host - account 三元组校验权限
can, err := canDoIbex(tpl.UpdateBy, tpl, host, targetCache, userCache)
if err != nil {
logger.Errorf("event_callback_ibex: check perm fail: %v", err)
return
}
if !can {
logger.Errorf("event_callback_ibex: user(%s) no permission", tpl.UpdateBy)
return
}
tagsMap := make(map[string]string)
for i := 0; i < len(event.TagsJSON); i++ {
pair := strings.TrimSpace(event.TagsJSON[i])
if pair == "" {
continue
}
arr := strings.Split(pair, "=")
if len(arr) != 2 {
continue
}
tagsMap[arr[0]] = arr[1]
}
// 附加告警级别 告警触发值标签
tagsMap["alert_severity"] = strconv.Itoa(event.Severity)
tagsMap["alert_trigger_value"] = event.TriggerValue
tags, err := json.Marshal(tagsMap)
if err != nil {
logger.Errorf("event_callback_ibex: failed to marshal tags to json: %v", tagsMap)
return
}
// call ibex
in := models.TaskForm{
Title: tpl.Title + " FH: " + host,
Account: tpl.Account,
Batch: tpl.Batch,
Tolerance: tpl.Tolerance,
Timeout: tpl.Timeout,
Pause: tpl.Pause,
Script: tpl.Script,
Args: tpl.Args,
Stdin: string(tags),
Action: "start",
Creator: tpl.UpdateBy,
Hosts: []string{host},
AlertTriggered: true,
}
id, err = TaskAdd(in, tpl.UpdateBy, ctx.IsCenter)
if err != nil {
logger.Errorf("event_callback_ibex: call ibex fail: %v", err)
return
}
// write db
record := models.TaskRecord{
Id: id,
EventId: event.Id,
GroupId: tpl.GroupId,
Title: in.Title,
Account: in.Account,
Batch: in.Batch,
Tolerance: in.Tolerance,
Timeout: in.Timeout,
Pause: in.Pause,
Script: in.Script,
Args: in.Args,
CreateAt: time.Now().Unix(),
CreateBy: in.Creator,
}
if err = record.Add(ctx); err != nil {
logger.Errorf("event_callback_ibex: persist task_record fail: %v", err)
}
}
func canDoIbex(username string, tpl *models.TaskTpl, host string, targetCache *memsto.TargetCacheType, userCache *memsto.UserCacheType) (bool, error) {
user := userCache.GetByUsername(username)
if user != nil && user.IsAdmin() {
return true, nil
}
target, has := targetCache.Get(host)
if !has {
return false, nil
}
return target.GroupId == tpl.GroupId, nil
}
func TaskAdd(f models.TaskForm, authUser string, isCenter bool) (int64, error) {
hosts := cleanHosts(f.Hosts)
if len(hosts) == 0 {
return 0, fmt.Errorf("arg(hosts) empty")
}
taskMeta := &imodels.TaskMeta{
Title: f.Title,
Account: f.Account,
Batch: f.Batch,
Tolerance: f.Tolerance,
Timeout: f.Timeout,
Pause: f.Pause,
Script: f.Script,
Args: f.Args,
Stdin: f.Stdin,
Creator: f.Creator,
}
err := taskMeta.CleanFields()
if err != nil {
return 0, err
}
taskMeta.HandleFH(hosts[0])
// 任务类型分为"告警规则触发"和"n9e center用户下发"两种;
// 边缘机房"告警规则触发"的任务不需要规划并且它可能是失联的无法使用db资源所以放入redis缓存中直接下发给agentd执行
if !isCenter && f.AlertTriggered {
if err := taskMeta.Create(); err != nil {
// 当网络不连通时生成唯一的id防止边缘机房中不同任务的id相同
// 方法是redis自增id去防止同一个机房的不同n9e edge生成的id相同
// 但没法防止不同边缘机房生成同样的id所以生成id的数据不会上报存入数据库只用于闭环执行。
taskMeta.Id, err = storage.IdGet()
if err != nil {
return 0, err
}
}
taskHost := imodels.TaskHost{
Id: taskMeta.Id,
Host: hosts[0],
Status: "running",
}
if err = taskHost.Create(); err != nil {
logger.Warningf("task_add_fail: authUser=%s title=%s err=%s", authUser, taskMeta.Title, err.Error())
}
// 缓存任务元信息和待下发的任务
err = taskMeta.Cache(hosts[0])
if err != nil {
return 0, err
}
} else {
// 如果是中心机房,还是保持之前的逻辑
err = taskMeta.Save(hosts, f.Action)
if err != nil {
return 0, err
}
}
logger.Infof("task_add_succ: authUser=%s title=%s", authUser, taskMeta.Title)
return taskMeta.Id, nil
}
func cleanHosts(formHosts []string) []string {
cnt := len(formHosts)
arr := make([]string, 0, cnt)
for i := 0; i < cnt; i++ {
item := strings.TrimSpace(formHosts[i])
if item == "" {
continue
}
if strings.HasPrefix(item, "#") {
continue
}
arr = append(arr, item)
}
return arr
}

View File

@@ -1,15 +1,9 @@
package sender
import (
"github.com/ccfos/nightingale/v6/models"
"html/template"
"strings"
"time"
"github.com/ccfos/nightingale/v6/alert/astats"
"github.com/ccfos/nightingale/v6/models"
"github.com/ccfos/nightingale/v6/pkg/poster"
"github.com/toolkits/pkg/logger"
)
type dingtalkMarkdown struct {
@@ -28,6 +22,10 @@ type dingtalk struct {
At dingtalkAt `json:"at"`
}
var (
_ CallBacker = (*DingtalkSender)(nil)
)
type DingtalkSender struct {
tpl *template.Template
}
@@ -72,6 +70,37 @@ func (ds *DingtalkSender) Send(ctx MessageContext) {
}
}
func (ds *DingtalkSender) CallBack(ctx CallBackContext) {
if len(ctx.Events) == 0 || len(ctx.CallBackURL) == 0 {
return
}
body := dingtalk{
Msgtype: "markdown",
Markdown: dingtalkMarkdown{
Title: ctx.Events[0].RuleName,
},
}
ats := ExtractAtsParams(ctx.CallBackURL)
message := BuildTplMessage(models.Dingtalk, ds.tpl, ctx.Events)
if len(ats) > 0 {
body.Markdown.Text = message + "\n@" + strings.Join(ats, "@")
body.At = dingtalkAt{
AtMobiles: ats,
IsAtAll: false,
}
} else {
// NoAt in url
body.Markdown.Text = message
}
doSend(ctx.CallBackURL, body, models.Dingtalk, ctx.Stats)
ctx.Stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
}
// extract urls and ats from Users
func (ds *DingtalkSender) extract(users []*models.User) ([]string, []string) {
urls := make([]string, 0, len(users))
@@ -91,15 +120,3 @@ func (ds *DingtalkSender) extract(users []*models.User) ([]string, []string) {
}
return urls, ats
}
func doSend(url string, body interface{}, channel string, stats *astats.Stats) {
stats.AlertNotifyTotal.WithLabelValues(channel).Inc()
res, code, err := poster.PostJSON(url, time.Second*5, body, 3)
if err != nil {
logger.Errorf("%s_sender: result=fail url=%s code=%d error=%v req:%v response=%s", channel, url, code, err, body, string(res))
stats.AlertNotifyErrorTotal.WithLabelValues(channel).Inc()
} else {
logger.Infof("%s_sender: result=succ url=%s code=%d req:%v response=%s", channel, url, code, body, string(res))
}
}

View File

@@ -1,6 +1,7 @@
package sender
import (
"fmt"
"html/template"
"strings"
@@ -22,10 +23,41 @@ type feishu struct {
At feishuAt `json:"at"`
}
var (
_ CallBacker = (*FeishuSender)(nil)
)
type FeishuSender struct {
tpl *template.Template
}
func (fs *FeishuSender) CallBack(ctx CallBackContext) {
if len(ctx.Events) == 0 || len(ctx.CallBackURL) == 0 {
return
}
ats := ExtractAtsParams(ctx.CallBackURL)
message := BuildTplMessage(models.Feishu, fs.tpl, ctx.Events)
if len(ats) > 0 {
atTags := ""
for _, at := range ats {
atTags += fmt.Sprintf("<at user_id=\"%s\"></at> ", at)
}
message = atTags + message
}
body := feishu{
Msgtype: "text",
Content: feishuContent{
Text: message,
},
}
doSend(ctx.CallBackURL, body, models.Feishu, ctx.Stats)
ctx.Stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
}
func (fs *FeishuSender) Send(ctx MessageContext) {
if len(ctx.Users) == 0 || len(ctx.Events) == 0 {
return

View File

@@ -3,6 +3,7 @@ package sender
import (
"fmt"
"html/template"
"net/url"
"strings"
"github.com/ccfos/nightingale/v6/models"
@@ -91,6 +92,37 @@ var (
}
)
func (fs *FeishuCardSender) CallBack(ctx CallBackContext) {
if len(ctx.Events) == 0 || len(ctx.CallBackURL) == 0 {
return
}
message := BuildTplMessage(models.FeishuCard, fs.tpl, ctx.Events)
color := "red"
lowerUnicode := strings.ToLower(message)
if strings.Count(lowerUnicode, Recovered) > 0 && strings.Count(lowerUnicode, Triggered) > 0 {
color = "orange"
} else if strings.Count(lowerUnicode, Recovered) > 0 {
color = "green"
}
SendTitle := fmt.Sprintf("🔔 %s", ctx.Events[0].RuleName)
body.Card.Header.Title.Content = SendTitle
body.Card.Header.Template = color
body.Card.Elements[0].Text.Content = message
body.Card.Elements[2].Elements[0].Content = SendTitle
// This is to be compatible with the feishucard interface, if with query string parameters, the request will fail
// Remove query parameters from the URL,
parsedURL, err := url.Parse(ctx.CallBackURL)
if err != nil {
return
}
parsedURL.RawQuery = ""
doSend(parsedURL.String(), body, models.FeishuCard, ctx.Stats)
}
func (fs *FeishuCardSender) Send(ctx MessageContext) {
if len(ctx.Users) == 0 || len(ctx.Events) == 0 {
return

265
alert/sender/ibex.go Normal file
View File

@@ -0,0 +1,265 @@
// @Author: Ciusyan 6/5/24
package sender
import (
"encoding/json"
"fmt"
"strconv"
"strings"
"time"
"github.com/ccfos/nightingale/v6/memsto"
"github.com/ccfos/nightingale/v6/models"
"github.com/ccfos/nightingale/v6/pkg/ctx"
imodels "github.com/flashcatcloud/ibex/src/models"
"github.com/flashcatcloud/ibex/src/storage"
"github.com/toolkits/pkg/logger"
)
var (
_ CallBacker = (*IbexCallBacker)(nil)
)
type IbexCallBacker struct {
targetCache *memsto.TargetCacheType
userCache *memsto.UserCacheType
taskTplCache *memsto.TaskTplCache
}
func (c *IbexCallBacker) CallBack(ctx CallBackContext) {
if len(ctx.CallBackURL) == 0 || len(ctx.Events) == 0 {
return
}
event := ctx.Events[0]
if event.IsRecovered {
return
}
c.handleIbex(ctx.Ctx, ctx.CallBackURL, event)
}
func (c *IbexCallBacker) handleIbex(ctx *ctx.Context, url string, event *models.AlertCurEvent) {
if imodels.DB() == nil && ctx.IsCenter {
logger.Warning("event_callback_ibex: db is nil")
return
}
arr := strings.Split(url, "/")
var idstr string
var host string
if len(arr) > 1 {
idstr = arr[1]
}
if len(arr) > 2 {
host = arr[2]
}
id, err := strconv.ParseInt(idstr, 10, 64)
if err != nil {
logger.Errorf("event_callback_ibex: failed to parse url: %s", url)
return
}
if host == "" {
// 用户在callback url中没有传入host就从event中解析
host = event.TargetIdent
}
if host == "" {
logger.Error("event_callback_ibex: failed to get host")
return
}
tpl := c.taskTplCache.Get(id)
if tpl == nil {
logger.Errorf("event_callback_ibex: no such tpl(%d)", id)
return
}
// check perm
// tpl.GroupId - host - account 三元组校验权限
can, err := canDoIbex(tpl.UpdateBy, tpl, host, c.targetCache, c.userCache)
if err != nil {
logger.Errorf("event_callback_ibex: check perm fail: %v", err)
return
}
if !can {
logger.Errorf("event_callback_ibex: user(%s) no permission", tpl.UpdateBy)
return
}
tagsMap := make(map[string]string)
for i := 0; i < len(event.TagsJSON); i++ {
pair := strings.TrimSpace(event.TagsJSON[i])
if pair == "" {
continue
}
arr := strings.Split(pair, "=")
if len(arr) != 2 {
continue
}
tagsMap[arr[0]] = arr[1]
}
// 附加告警级别 告警触发值标签
tagsMap["alert_severity"] = strconv.Itoa(event.Severity)
tagsMap["alert_trigger_value"] = event.TriggerValue
tags, err := json.Marshal(tagsMap)
if err != nil {
logger.Errorf("event_callback_ibex: failed to marshal tags to json: %v", tagsMap)
return
}
// call ibex
in := models.TaskForm{
Title: tpl.Title + " FH: " + host,
Account: tpl.Account,
Batch: tpl.Batch,
Tolerance: tpl.Tolerance,
Timeout: tpl.Timeout,
Pause: tpl.Pause,
Script: tpl.Script,
Args: tpl.Args,
Stdin: string(tags),
Action: "start",
Creator: tpl.UpdateBy,
Hosts: []string{host},
AlertTriggered: true,
}
id, err = TaskAdd(in, tpl.UpdateBy, ctx.IsCenter)
if err != nil {
logger.Errorf("event_callback_ibex: call ibex fail: %v", err)
return
}
// write db
record := models.TaskRecord{
Id: id,
EventId: event.Id,
GroupId: tpl.GroupId,
Title: in.Title,
Account: in.Account,
Batch: in.Batch,
Tolerance: in.Tolerance,
Timeout: in.Timeout,
Pause: in.Pause,
Script: in.Script,
Args: in.Args,
CreateAt: time.Now().Unix(),
CreateBy: in.Creator,
}
if err = record.Add(ctx); err != nil {
logger.Errorf("event_callback_ibex: persist task_record fail: %v", err)
}
}
func canDoIbex(username string, tpl *models.TaskTpl, host string, targetCache *memsto.TargetCacheType, userCache *memsto.UserCacheType) (bool, error) {
user := userCache.GetByUsername(username)
if user != nil && user.IsAdmin() {
return true, nil
}
target, has := targetCache.Get(host)
if !has {
return false, nil
}
return target.GroupId == tpl.GroupId, nil
}
func TaskAdd(f models.TaskForm, authUser string, isCenter bool) (int64, error) {
hosts := cleanHosts(f.Hosts)
if len(hosts) == 0 {
return 0, fmt.Errorf("arg(hosts) empty")
}
taskMeta := &imodels.TaskMeta{
Title: f.Title,
Account: f.Account,
Batch: f.Batch,
Tolerance: f.Tolerance,
Timeout: f.Timeout,
Pause: f.Pause,
Script: f.Script,
Args: f.Args,
Stdin: f.Stdin,
Creator: f.Creator,
}
err := taskMeta.CleanFields()
if err != nil {
return 0, err
}
taskMeta.HandleFH(hosts[0])
// 任务类型分为"告警规则触发"和"n9e center用户下发"两种;
// 边缘机房"告警规则触发"的任务不需要规划并且它可能是失联的无法使用db资源所以放入redis缓存中直接下发给agentd执行
if !isCenter && f.AlertTriggered {
if err := taskMeta.Create(); err != nil {
// 当网络不连通时生成唯一的id防止边缘机房中不同任务的id相同
// 方法是redis自增id去防止同一个机房的不同n9e edge生成的id相同
// 但没法防止不同边缘机房生成同样的id所以生成id的数据不会上报存入数据库只用于闭环执行。
taskMeta.Id, err = storage.IdGet()
if err != nil {
return 0, err
}
}
taskHost := imodels.TaskHost{
Id: taskMeta.Id,
Host: hosts[0],
Status: "running",
}
if err = taskHost.Create(); err != nil {
logger.Warningf("task_add_fail: authUser=%s title=%s err=%s", authUser, taskMeta.Title, err.Error())
}
// 缓存任务元信息和待下发的任务
err = taskMeta.Cache(hosts[0])
if err != nil {
return 0, err
}
} else {
// 如果是中心机房,还是保持之前的逻辑
err = taskMeta.Save(hosts, f.Action)
if err != nil {
return 0, err
}
}
logger.Infof("task_add_succ: authUser=%s title=%s", authUser, taskMeta.Title)
return taskMeta.Id, nil
}
func cleanHosts(formHosts []string) []string {
cnt := len(formHosts)
arr := make([]string, 0, cnt)
for i := 0; i < cnt; i++ {
item := strings.TrimSpace(formHosts[i])
if item == "" {
continue
}
if strings.HasPrefix(item, "#") {
continue
}
arr = append(arr, item)
}
return arr
}

View File

@@ -45,6 +45,21 @@ func (ms *MmSender) Send(ctx MessageContext) {
})
}
func (ms *MmSender) CallBack(ctx CallBackContext) {
if len(ctx.Events) == 0 || len(ctx.CallBackURL) == 0 {
return
}
message := BuildTplMessage(models.Mm, ms.tpl, ctx.Events)
SendMM(MatterMostMessage{
Text: message,
Tokens: []string{ctx.CallBackURL},
Stats: ctx.Stats,
})
ctx.Stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
}
func (ms *MmSender) extract(users []*models.User) []string {
tokens := make([]string, 0, len(users))
for _, user := range users {

View File

@@ -21,10 +21,29 @@ type telegram struct {
Text string `json:"text"`
}
var (
_ CallBacker = (*TelegramSender)(nil)
)
type TelegramSender struct {
tpl *template.Template
}
func (ts *TelegramSender) CallBack(ctx CallBackContext) {
if len(ctx.Events) == 0 || len(ctx.CallBackURL) == 0 {
return
}
message := BuildTplMessage(models.Telegram, ts.tpl, ctx.Events)
SendTelegram(TelegramMessage{
Text: message,
Tokens: []string{ctx.CallBackURL},
Stats: ctx.Stats,
})
ctx.Stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
}
func (ts *TelegramSender) Send(ctx MessageContext) {
if len(ctx.Users) == 0 || len(ctx.Events) == 0 {
return

View File

@@ -16,10 +16,31 @@ type wecom struct {
Markdown wecomMarkdown `json:"markdown"`
}
var (
_ CallBacker = (*WecomSender)(nil)
)
type WecomSender struct {
tpl *template.Template
}
func (ws *WecomSender) CallBack(ctx CallBackContext) {
if len(ctx.Events) == 0 || len(ctx.CallBackURL) == 0 {
return
}
message := BuildTplMessage(models.Wecom, ws.tpl, ctx.Events)
body := wecom{
Msgtype: "markdown",
Markdown: wecomMarkdown{
Content: message,
},
}
doSend(ctx.CallBackURL, body, models.Wecom, ctx.Stats)
ctx.Stats.AlertNotifyTotal.WithLabelValues("rule_callback").Inc()
}
func (ws *WecomSender) Send(ctx MessageContext) {
if len(ctx.Users) == 0 || len(ctx.Events) == 0 {
return

View File

@@ -76,6 +76,8 @@ ops:
- "/dashboards/add"
- "/dashboards/put"
- "/dashboards/del"
- "/embedded-dashboards/put"
- "/embedded-dashboards"
- name: alert
cname: 告警规则

View File

@@ -93,9 +93,16 @@ func (rt *Router) alertHisEventGet(c *gin.Context) {
func GetBusinessGroupIds(c *gin.Context, ctx *ctx.Context, eventHistoryGroupView bool) ([]int64, error) {
bgid := ginx.QueryInt64(c, "bgid", 0)
var bgids []int64
user := c.MustGet("user").(*models.User)
if !eventHistoryGroupView || user.IsAdmin() {
if !eventHistoryGroupView || strings.HasPrefix(c.Request.URL.Path, "/v1") {
if bgid > 0 {
return []int64{bgid}, nil
}
return bgids, nil
}
user := c.MustGet("user").(*models.User)
if user.IsAdmin() {
if bgid > 0 {
return []int64{bgid}, nil
}

View File

@@ -42,7 +42,7 @@ func (rt *Router) alertRuleGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -44,7 +44,7 @@ func (rt *Router) alertSubscribeGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -272,7 +272,7 @@ func (rt *Router) boardGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -8,6 +8,7 @@ import (
"github.com/gin-gonic/gin"
"github.com/toolkits/pkg/ginx"
"github.com/toolkits/pkg/i18n"
)
// single or import
@@ -30,7 +31,7 @@ func (rt *Router) builtinMetricsAdd(c *gin.Context) {
lst[i].Lang = lang
lst[i].UUID = time.Now().UnixNano()
if err := lst[i].Add(rt.Ctx, username); err != nil {
reterr[lst[i].Name] = err.Error()
reterr[lst[i].Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
}
}
ginx.NewRender(c).Data(reterr, nil)

View File

@@ -9,6 +9,7 @@ import (
"github.com/ccfos/nightingale/v6/models"
"github.com/gin-gonic/gin"
"github.com/toolkits/pkg/ginx"
"github.com/toolkits/pkg/i18n"
)
type Board struct {
@@ -63,7 +64,7 @@ func (rt *Router) builtinPayloadsAdd(c *gin.Context) {
}
if err := bp.Add(rt.Ctx, username); err != nil {
reterr[bp.Name] = err.Error()
reterr[bp.Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
}
}
continue
@@ -92,7 +93,7 @@ func (rt *Router) builtinPayloadsAdd(c *gin.Context) {
}
if err := bp.Add(rt.Ctx, username); err != nil {
reterr[bp.Name] = err.Error()
reterr[bp.Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
}
} else if lst[i].Type == "dashboard" {
if strings.HasPrefix(strings.TrimSpace(lst[i].Content), "[") {
@@ -126,7 +127,7 @@ func (rt *Router) builtinPayloadsAdd(c *gin.Context) {
}
if err := bp.Add(rt.Ctx, username); err != nil {
reterr[bp.Name] = err.Error()
reterr[bp.Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
}
}
continue
@@ -134,7 +135,7 @@ func (rt *Router) builtinPayloadsAdd(c *gin.Context) {
dashboard := Board{}
if err := json.Unmarshal([]byte(lst[i].Content), &dashboard); err != nil {
reterr[lst[i].Name] = err.Error()
reterr[lst[i].Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
continue
}
@@ -155,11 +156,11 @@ func (rt *Router) builtinPayloadsAdd(c *gin.Context) {
}
if err := bp.Add(rt.Ctx, username); err != nil {
reterr[bp.Name] = err.Error()
reterr[bp.Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
}
} else {
if err := lst[i].Add(rt.Ctx, username); err != nil {
reterr[lst[i].Name] = err.Error()
reterr[lst[i].Name] = i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
}
}

View File

@@ -262,6 +262,15 @@ func (rt *Router) loginCallback(c *gin.Context) {
user.FullSsoFields("oidc", ret.Username, ret.Nickname, ret.Phone, ret.Email, rt.Sso.OIDC.DefaultRoles)
// create user from oidc
ginx.Dangerous(user.Add(rt.Ctx))
if len(rt.Sso.OIDC.DefaultTeams) > 0 {
for _, gid := range rt.Sso.OIDC.DefaultTeams {
err = models.UserGroupMemberAdd(rt.Ctx, gid, user.Id)
if err != nil {
logger.Errorf("user:%v UserGroupMemberAdd: %s", user, err)
}
}
}
}
// set user login state

View File

@@ -35,7 +35,7 @@ func (rt *Router) alertMuteGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -34,7 +34,7 @@ func (rt *Router) recordingRuleGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -1,7 +1,6 @@
package router
import (
"net/http"
"time"
"github.com/ccfos/nightingale/v6/alert/sender"
@@ -53,7 +52,7 @@ func (rt *Router) taskGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -47,7 +47,7 @@ func (rt *Router) taskTplGetsByGids(c *gin.Context) {
ginx.Dangerous(err)
if len(gids) == 0 {
ginx.Bomb(http.StatusForbidden, "forbidden")
ginx.NewRender(c).Data([]int{}, nil)
return
}
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/gin-gonic/gin"
"github.com/toolkits/pkg/ginx"
"github.com/toolkits/pkg/logger"
"github.com/toolkits/pkg/str"
)
func (rt *Router) checkBusiGroupPerm(c *gin.Context) {
@@ -31,8 +32,36 @@ func (rt *Router) userGroupGets(c *gin.Context) {
}
func (rt *Router) userGroupGetsByService(c *gin.Context) {
lst, err := models.UserGroupGetAll(rt.Ctx)
ginx.NewRender(c).Data(lst, err)
ids := str.IdsInt64(ginx.QueryStr(c, "ids", ""))
if len(ids) == 0 {
lst, err := models.UserGroupGetAll(rt.Ctx)
ginx.Dangerous(err)
for i := 0; i < len(lst); i++ {
ids, err := models.MemberIds(rt.Ctx, lst[i].Id)
ginx.Dangerous(err)
lst[i].Users, err = models.UserGetsByIds(rt.Ctx, ids)
ginx.Dangerous(err)
}
ginx.NewRender(c).Data(lst, err)
return
}
lst := make([]models.UserGroup, 0)
for _, id := range ids {
ug := UserGroup(rt.Ctx, id)
ids, err := models.MemberIds(rt.Ctx, ug.Id)
ginx.Dangerous(err)
ug.Users, err = models.UserGetsByIds(rt.Ctx, ids)
ginx.Dangerous(err)
lst = append(lst, *ug)
}
ginx.NewRender(c).Data(lst, nil)
}
// user group member get by service

1777
docker/initsql/sqlite.sql Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -75,10 +75,11 @@ OpenRSA = false
[DB]
# postgres: host=%s port=%s user=%s dbname=%s password=%s sslmode=%s
# postgres: DSN="host=127.0.0.1 port=5432 user=root dbname=n9e_v6 password=1234 sslmode=disable"
DSN="root:1234@tcp(127.0.0.1:3306)/n9e_v6?charset=utf8mb4&parseTime=True&loc=Local&allowNativePasswords=true"
# sqlite: DSN="/path/to/filename.db"
DSN = "root:1234@tcp(127.0.0.1:3306)/n9e_v6?charset=utf8mb4&parseTime=True&loc=Local&allowNativePasswords=true"
# enable debug mode or not
Debug = false
# mysql postgres
# mysql postgres sqlite
DBType = "mysql"
# unit: s
MaxLifetime = 7200

6
go.mod
View File

@@ -8,7 +8,7 @@ require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/dgrijalva/jwt-go v3.2.0+incompatible
github.com/expr-lang/expr v1.16.1
github.com/flashcatcloud/ibex v1.3.3
github.com/flashcatcloud/ibex v1.3.4
github.com/gin-contrib/pprof v1.4.0
github.com/gin-gonic/gin v1.9.1
github.com/go-ldap/ldap/v3 v3.4.4
@@ -39,7 +39,8 @@ require (
gopkg.in/yaml.v2 v2.4.0
gorm.io/driver/mysql v1.4.4
gorm.io/driver/postgres v1.4.5
gorm.io/gorm v1.24.2
gorm.io/driver/sqlite v1.5.5
gorm.io/gorm v1.25.7-0.20240204074919-46816ad31dde
)
require (
@@ -77,6 +78,7 @@ require (
github.com/josharian/intern v1.0.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.4 // indirect
github.com/leodido/go-urn v1.2.4 // indirect
github.com/mattn/go-sqlite3 v1.14.17 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect

12
go.sum
View File

@@ -47,8 +47,8 @@ github.com/fatih/camelcase v1.0.0 h1:hxNvNX/xYBp0ovncs8WyWZrOrpBNub/JfaMvbURyft8
github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc=
github.com/fatih/structs v1.1.0 h1:Q7juDM0QtcnhCpeyLGQKyg4TOIghuNXrkL32pHAUMxo=
github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
github.com/flashcatcloud/ibex v1.3.3 h1:1Bxk5sgpsq4+e9bMchucGttg8Sw4KEpZy9tctFfj/cE=
github.com/flashcatcloud/ibex v1.3.3/go.mod h1:T8hbMUySK2q6cXUaYp0AUVeKkU9Od2LjzwmB5lmTRBM=
github.com/flashcatcloud/ibex v1.3.4 h1:s5MgQmDIYR18liBKPNl96kC/h1jOTZjIOlUWeSx0710=
github.com/flashcatcloud/ibex v1.3.4/go.mod h1:T8hbMUySK2q6cXUaYp0AUVeKkU9Od2LjzwmB5lmTRBM=
github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU=
github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA=
github.com/garyburd/redigo v1.6.2/go.mod h1:NR3MbYisc3/PwhQ00EMzDiPmrwpPxAn5GI05/YaO1SY=
@@ -209,6 +209,8 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.17 h1:mCRHCLDUBXgpKAqIKsaAaAsrAlbkeomtRFKXh2L6YIM=
github.com/mattn/go-sqlite3 v1.14.17/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -464,9 +466,11 @@ gorm.io/driver/mysql v1.4.4 h1:MX0K9Qvy0Na4o7qSC/YI7XxqUw5KDw01umqgID+svdQ=
gorm.io/driver/mysql v1.4.4/go.mod h1:BCg8cKI+R0j/rZRQxeKis/forqRwRSYOR8OM3Wo6hOM=
gorm.io/driver/postgres v1.4.5 h1:mTeXTTtHAgnS9PgmhN2YeUbazYpLhUI1doLnw42XUZc=
gorm.io/driver/postgres v1.4.5/go.mod h1:GKNQYSJ14qvWkvPwXljMGehpKrhlDNsqYRr5HnYGncg=
gorm.io/driver/sqlite v1.5.5 h1:7MDMtUZhV065SilG62E0MquljeArQZNfJnjd9i9gx3E=
gorm.io/driver/sqlite v1.5.5/go.mod h1:6NgQ7sQWAIFsPrJJl1lSNSu2TABh0ZZ/zm5fosATavE=
gorm.io/gorm v1.23.8/go.mod h1:l2lP/RyAtc1ynaTjFksBde/O8v9oOGIApu2/xRitmZk=
gorm.io/gorm v1.24.1-0.20221019064659-5dd2bb482755/go.mod h1:DVrVomtaYTbqs7gB/x2uVvqnXzv0nqjB396B8cG4dBA=
gorm.io/gorm v1.24.2 h1:9wR6CFD+G8nOusLdvkZelOEhpJVwwHzpQOUM+REd6U0=
gorm.io/gorm v1.24.2/go.mod h1:DVrVomtaYTbqs7gB/x2uVvqnXzv0nqjB396B8cG4dBA=
gorm.io/gorm v1.25.7-0.20240204074919-46816ad31dde h1:9DShaph9qhkIYw7QF91I/ynrr4cOO2PZra2PFD7Mfeg=
gorm.io/gorm v1.25.7-0.20240204074919-46816ad31dde/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=

View File

@@ -134,6 +134,7 @@ func (e *AlertCurEvent) ParseRule(field string) error {
var defs = []string{
"{{$labels := .TagsMap}}",
"{{$value := .TriggerValue}}",
"{{$annotations := .AnnotationsJSON}}",
}
text := strings.Join(append(defs, f), "")

View File

@@ -394,6 +394,10 @@ func (s *AlertSubscribe) ModifyEvent(event *AlertCurEvent) {
if s.RedefineWebhooks == 1 {
event.Callbacks = s.Webhooks
event.CallbacksJSON = s.WebhooksJson
} else {
// 将 callback 重置为空,防止事件被订阅之后,再次将事件发送给回调地址
event.Callbacks = ""
event.CallbacksJSON = []string{}
}
event.NotifyGroups = s.UserGroupIds

View File

@@ -102,6 +102,15 @@ func (b *Board) Add(ctx *ctx.Context) error {
}
}
cnt, err := Count(DB(ctx).Model(b).Where("name = ? and group_id = ?", b.Name, b.GroupId))
if err != nil {
return err
}
if cnt > 0 {
return errors.New("Name duplicate")
}
now := time.Now().Unix()
b.CreateAt = now
b.UpdateAt = now

View File

@@ -15,7 +15,7 @@ type BuiltinMetric struct {
UUID int64 `json:"uuid" gorm:"type:bigint;not null;default:0;comment:'uuid'"`
Collector string `json:"collector" gorm:"type:varchar(191);not null;index:idx_collector,sort:asc;comment:'type of collector'"` // Type of collector (e.g., 'categraf', 'telegraf')
Typ string `json:"typ" gorm:"type:varchar(191);not null;index:idx_typ,sort:asc;comment:'type of metric'"` // Type of metric (e.g., 'host', 'mysql', 'redis')
Name string `json:"name" gorm:"type:varchar(191);not null;index:idx_name,sort:asc;comment:'name of metric'"`
Name string `json:"name" gorm:"type:varchar(191);not null;index:idx_builtinmetric_name,sort:asc;comment:'name of metric'"`
Unit string `json:"unit" gorm:"type:varchar(191);not null;comment:'unit of metric'"`
Note string `json:"note" gorm:"type:varchar(4096);not null;comment:'description of metric'"`
Lang string `json:"lang" gorm:"type:varchar(191);not null;default:'zh';index:idx_lang,sort:asc;comment:'language'"`

View File

@@ -10,7 +10,7 @@ import (
type MetricFilter struct {
ID int64 `json:"id" gorm:"primaryKey;type:bigint;autoIncrement;comment:'unique identifier'"`
Name string `json:"name" gorm:"type:varchar(191);not null;index:idx_name,sort:asc;comment:'name of metric filter'"`
Name string `json:"name" gorm:"type:varchar(191);not null;index:idx_metricfilter_name,sort:asc;comment:'name of metric filter'"`
Configs string `json:"configs" gorm:"type:varchar(4096);not null;comment:'configuration of metric filter'"`
GroupsPerm []GroupPerm `json:"groups_perm" gorm:"type:text;serializer:json;"`
CreateAt int64 `json:"create_at" gorm:"type:bigint;not null;default:0;comment:'create time'"`

View File

@@ -13,7 +13,7 @@ type BuiltinPayload struct {
Type string `json:"type" gorm:"type:varchar(191);not null;index:idx_type,sort:asc;comment:'type of payload'"` // Alert Dashboard Collet
Component string `json:"component" gorm:"type:varchar(191);not null;index:idx_component,sort:asc;comment:'component of payload'"` // Host MySQL Redis
Cate string `json:"cate" gorm:"type:varchar(191);not null;comment:'category of payload'"` // categraf_v1 telegraf_v1
Name string `json:"name" gorm:"type:varchar(191);not null;index:idx_name,sort:asc;comment:'name of payload'"` //
Name string `json:"name" gorm:"type:varchar(191);not null;index:idx_buildinpayload_name,sort:asc;comment:'name of payload'"` //
Tags string `json:"tags" gorm:"type:varchar(191);not null;default:'';comment:'tags of payload'"` // {"host":"
Content string `json:"content" gorm:"type:longtext;not null;comment:'content of payload'"`
UUID int64 `json:"uuid" gorm:"type:bigint;not null;index:idx_uuid;comment:'uuid of payload'"`

View File

@@ -19,6 +19,17 @@ func Migrate(db *gorm.DB) {
}
func MigrateIbexTables(db *gorm.DB) {
var tableOptions string
switch db.Dialector.(type) {
case *mysql.Dialector:
tableOptions = "ENGINE=InnoDB DEFAULT CHARSET=utf8mb4"
case *postgres.Dialector:
tableOptions = "ENCODING='UTF8'"
}
if tableOptions != "" {
db = db.Set("gorm:table_options", tableOptions)
}
dts := []interface{}{&imodels.TaskMeta{}, &imodels.TaskScheduler{}, &imodels.TaskSchedulerHealth{}, &imodels.TaskHostDoing{}, &imodels.TaskAction{}}
for _, dt := range dts {
err := db.AutoMigrate(dt)

View File

@@ -33,6 +33,16 @@ const (
FeishuKey = "feishu_robot_token"
MmKey = "mm_webhook_url"
TelegramKey = "telegram_robot_token"
DingtalkDomain = "oapi.dingtalk.com"
WecomDomain = "qyapi.weixin.qq.com"
FeishuDomain = "open.feishu.cn"
// FeishuCardDomain The domain name of the feishu card is the same as the feishu,distinguished by the parameter
FeishuCardDomain = "open.feishu.cn?card=1"
TelegramDomain = "api.telegram.org"
IbexDomain = "ibex"
DefaultDomain = "default"
)
var (

View File

@@ -20,6 +20,7 @@ type UserGroup struct {
UpdateAt int64 `json:"update_at"`
UpdateBy string `json:"update_by"`
UserIds []int64 `json:"-" gorm:"-"`
Users []User `json:"users" gorm:"-"`
}
func (ug *UserGroup) TableName() string {

View File

@@ -20,8 +20,8 @@ var I18N = `
"invalid tagkey(%s)": "tagkey不合法[%s]",
"duplicate tagkey(%s)":"tagkey(%s)重复了",
"name is empty": "名称不能为空",
"Ident duplicate":"盘唯一标识已存在",
"No such dashboard":"盘不存在",
"Ident duplicate":"仪表盘唯一标识已存在",
"No such dashboard":"仪表盘不存在",
"Name has invalid characters":"名称包含非法字符",
"Name is blank":"名称不能为空",
"forbidden":"没有权限",
@@ -48,7 +48,60 @@ var I18N = `
"invalid ibex address: %s":"ibex %s 地址无效",
"url path invalid":"url非法",
"no such server":"无此实例",
"admin role can not be modified":"管理员角色不允许修改"
"admin role can not be modified":"管理员角色不允许修改",
"builtin payload already exists":"内置模板已存在"
},
"zh_CN": {
"Username or password invalid": "用户名或密码错误",
"incorrect verification code": "验证码错误",
"roles empty": "角色不能为空",
"Username already exists": "此用户名已存在 请使用其他用户名",
"failed to count user-groups": "校验数据失败 请重试",
"UserGroup already exists": "组名已存在 请使用其他名称",
"members empty": "成员不能为空",
"At least one team have rw permission": "至少需要有一个团队有读写权限",
"Failed to create BusiGroup(%s)": "[%s]创建失败 请重试",
"business group id invalid": "业务组 id 不正确",
"idents empty": "监控对象不能为空",
"invalid tag(%s)": "tag不合法[%s]",
"invalid tagkey(%s): cannot contains . ": "tagkey[%s]不能包含.",
"invalid tagkey(%s): cannot contains _ ": "tagkey[%s]不能包含_",
"invalid tagkey(%s)": "tagkey不合法[%s]",
"duplicate tagkey(%s)":"tagkey(%s)重复了",
"name is empty": "名称不能为空",
"Ident duplicate":"仪表盘唯一标识已存在",
"Name duplicate":"仪表盘名称已存在",
"No such dashboard":"仪表盘不存在",
"Name has invalid characters":"名称包含非法字符",
"Name is blank":"名称不能为空",
"forbidden":"没有权限",
"builtin alerts is empty, file: %s":"内置告警模板为空 %s",
"input json is empty":"提交内容不能为空",
"fields empty":"选择字段不能为空",
"No such AlertRule":"无此告警规则",
"GroupId(%d) invalid":"业务组id无效",
"No such recording rule":"无此记录规则",
"tags is blank":"标签不能为空",
"oops... etime(%d) <= btime(%d)":"开始时间,不能大于结束时间",
"group_id invalid":"业务组无效",
"No such AlertMute":"无此屏蔽规则",
"rule_id and tags are both blank":"告警规则和标签不能同时为空",
"rule is blank":"规则不能为空",
"rule invalid":"规则无效 请检查是否正确",
"unsupported field: %s":"不支持字段 %s",
"arg(batch) should be nonnegative":"batch 不能为负数",
"arg(tolerance) should be nonnegative":"tolerance 不能为负数",
"arg(timeout) should be nonnegative":"timeout 不能为负数",
"arg(timeout) longer than five days":"timeout 时间不能超过5天",
"arg(title) is required":"title 为必填项",
"created task.id is zero":"任务id为零",
"invalid ibex address: %s":"ibex %s 地址无效",
"url path invalid":"url非法",
"no such server":"无此实例",
"admin role can not be modified":"管理员角色不允许修改",
"builtin payload already exists":"内置模板已存在",
"builtin metric already exists":"内置指标已存在",
"AlertRule already exists":"告警规则已存在"
}
}
`

View File

@@ -32,6 +32,7 @@ type SsoClient struct {
Email string
}
DefaultRoles []string
DefaultTeams []int64
Ctx context.Context
Provider *oidc.Provider
@@ -55,6 +56,7 @@ type Config struct {
Email string
}
DefaultRoles []string
DefaultTeams []int64
Scopes []string
}
@@ -90,6 +92,7 @@ func (s *SsoClient) Reload(cf Config) error {
s.Attributes.Email = cf.Attributes.Email
s.DisplayName = cf.DisplayName
s.DefaultRoles = cf.DefaultRoles
s.DefaultTeams = cf.DefaultTeams
s.Ctx = context.Background()
if cf.SkipTlsVerify {

View File

@@ -9,6 +9,7 @@ import (
tklog "github.com/toolkits/pkg/logger"
"gorm.io/driver/mysql"
"gorm.io/driver/postgres"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
"gorm.io/gorm/schema"
@@ -72,12 +73,16 @@ func (l *TKitLogger) Printf(s string, i ...interface{}) {
// New Create gorm.DB instance
func New(c DBConfig) (*gorm.DB, error) {
var dialector gorm.Dialector
sqliteUsed := false
switch strings.ToLower(c.DBType) {
case "mysql":
dialector = mysql.Open(c.DSN)
case "postgres":
dialector = postgres.Open(c.DSN)
case "sqlite":
dialector = sqlite.Open(c.DSN)
sqliteUsed = true
default:
return nil, fmt.Errorf("dialector(%s) not supported", c.DBType)
}
@@ -104,9 +109,11 @@ func New(c DBConfig) (*gorm.DB, error) {
return nil, err
}
sqlDB.SetMaxIdleConns(c.MaxIdleConns)
sqlDB.SetMaxOpenConns(c.MaxOpenConns)
sqlDB.SetConnMaxLifetime(time.Duration(c.MaxLifetime) * time.Second)
if !sqliteUsed {
sqlDB.SetMaxIdleConns(c.MaxIdleConns)
sqlDB.SetMaxOpenConns(c.MaxOpenConns)
sqlDB.SetConnMaxLifetime(time.Duration(c.MaxLifetime) * time.Second)
}
return db, nil
}