mirror of
https://github.com/ccfos/nightingale.git
synced 2026-03-04 23:18:57 +00:00
Compare commits
1 Commits
homeimg
...
optimize-c
| Author | SHA1 | Date | |
|---|---|---|---|
|
f6a857f030 |
105
README.md
105
README.md
@@ -3,7 +3,7 @@
|
||||
<img src="doc/img/Nightingale_L_V.png" alt="nightingale - cloud native monitoring" width="100" /></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<b>Open-Source Alerting Expert</b>
|
||||
<b>开源告警管理专家</b>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
@@ -25,91 +25,94 @@
|
||||
|
||||
|
||||
|
||||
[English](./README.md) | [中文](./README_zh.md)
|
||||
[English](./README_en.md) | [中文](./README.md)
|
||||
|
||||
## 🎯 What is Nightingale
|
||||
## 夜莺是什么
|
||||
|
||||
Nightingale is an open-source monitoring project that focuses on alerting. Similar to Grafana, Nightingale also connects with various existing data sources. However, while Grafana emphasizes visualization, Nightingale places greater emphasis on the alerting engine, as well as the processing and distribution of alarms.
|
||||
夜莺监控（Nightingale）是一款侧重告警的监控类开源项目。类似 Grafana 的数据源集成方式，夜莺也是对接多种既有的数据源，不过 Grafana 侧重在可视化，夜莺是侧重在告警引擎、告警事件的处理和分发。
|
||||
|
||||
> The Nightingale project was initially developed and open-sourced by DiDi.inc. On May 11, 2022, it was donated to the Open Source Development Committee of the China Computer Federation (CCF ODC).
|
||||
夜莺监控项目，最初由滴滴开发和开源，并于 2022 年 5 月 11 日，捐赠予中国计算机学会开源发展委员会（CCF ODC），为 CCF ODC 成立后接受捐赠的第一个开源项目。
|
||||
|
||||
|
||||
## 夜莺的工作逻辑
|
||||
|
||||
## 💡 How Nightingale Works
|
||||
很多用户已经自行采集了指标、日志数据，此时就把存储库（VictoriaMetrics、ElasticSearch等）作为数据源接入夜莺，即可在夜莺里配置告警规则、通知规则，完成告警事件的生成和派发。
|
||||
|
||||
Many users have already collected metrics and log data. In this case, you can connect your storage repositories (such as VictoriaMetrics, ElasticSearch, etc.) as data sources in Nightingale. This allows you to configure alerting rules and notification rules within Nightingale, enabling the generation and distribution of alarms.
|
||||
|
||||
|
||||
|
||||
夜莺项目本身不提供监控数据采集能力。推荐您使用 [Categraf](https://github.com/flashcatcloud/categraf) 作为采集器，可以和夜莺丝滑对接。
|
||||
|
||||
Nightingale itself does not provide monitoring data collection capabilities. We recommend using [Categraf](https://github.com/flashcatcloud/categraf) as the collector, which integrates seamlessly with Nightingale.
|
||||
[Categraf](https://github.com/flashcatcloud/categraf) 可以采集操作系统、网络设备、各类中间件、数据库的监控数据，通过 Remote Write 协议推送给夜莺，夜莺把监控数据转存到时序库（如 Prometheus、VictoriaMetrics 等），并提供告警和可视化能力。
|
||||
|
||||
[Categraf](https://github.com/flashcatcloud/categraf) can collect monitoring data from operating systems, network devices, various middleware, and databases. It pushes this data to Nightingale via the `Prometheus Remote Write` protocol. Nightingale then stores the monitoring data in a time-series database (such as Prometheus, VictoriaMetrics, etc.) and provides alerting and visualization capabilities.
|
||||
对于个别边缘机房，如果和中心夜莺服务端网络链路不好，希望提升告警可用性，夜莺也提供边缘机房告警引擎下沉部署模式，这个模式下，即便边缘和中心端网络割裂，告警功能也不受影响。
|
||||
|
||||
For certain edge data centers with poor network connectivity to the central Nightingale server, we offer a distributed deployment mode for the alerting engine. In this mode, even if the network is disconnected, the alerting functionality remains unaffected.
|
||||
|
||||
|
||||
|
||||
> 上图中，机房A和中心机房的网络链路很好，所以直接由中心端的夜莺进程做告警引擎，机房B和中心机房的网络链路不好，所以在机房B部署了 `n9e-edge` 做告警引擎，对机房B的数据源做告警判定。
|
||||
|
||||
> In the above diagram, Data Center A has a good network with the central data center, so it uses the Nightingale process in the central data center as the alerting engine. Data Center B has a poor network with the central data center, so it deploys `n9e-edge` as the alerting engine to handle alerting for its own data sources.
|
||||
## 告警降噪、升级、协同
|
||||
|
||||
## 🔕 Alert Noise Reduction, Escalation, and Collaboration
|
||||
夜莺的侧重点是做告警引擎，即负责产生告警事件，并根据规则做灵活派发，内置支持 20 种通知媒介（电话、短信、邮件、钉钉、飞书、企微、Slack 等）。
|
||||
|
||||
Nightingale focuses on being an alerting engine, responsible for generating alarms and flexibly distributing them based on rules. It supports 20 built-in notification medias (such as phone calls, SMS, email, DingTalk, Slack, etc.).
|
||||
如果您有更高级的需求，比如：
|
||||
|
||||
If you have more advanced requirements, such as:
|
||||
- Want to consolidate events from multiple monitoring systems into one platform for unified noise reduction, response handling, and data analysis.
|
||||
- Want to support personnel scheduling, practice on-call culture, and support alert escalation (to avoid missing alerts) and collaborative handling.
|
||||
- 想要把公司的多套监控系统产生的事件聚拢到一个平台，统一做收敛降噪、响应处理、数据分析
|
||||
- 想要支持人员的排班，践行 On-call 文化，想要支持告警认领、升级（避免遗漏）、协同处理
|
||||
|
||||
Then Nightingale is not suitable. It is recommended that you choose on-call products such as PagerDuty and FlashDuty. These products are simple and easy to use.
|
||||
那夜莺是不合适的，推荐您选用 [FlashDuty](https://flashcat.cloud/product/flashcat-duty/) 这样的 On-call 产品，产品简单易用，也有免费套餐。
|
||||
|
||||
## 🗨️ Communication Channels
|
||||
|
||||
- **Report Bugs:** It is highly recommended to submit issues via the [Nightingale GitHub Issue tracker](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=kind%2Fbug&projects=&template=bug_report.yml).
|
||||
- **Documentation:** For more information, we recommend thoroughly browsing the [Nightingale Documentation Site](https://n9e.github.io/).
|
||||
## 相关资料 & 交流渠道
|
||||
- 📚 [夜莺介绍PPT](https://mp.weixin.qq.com/s/Mkwx_46xrltSq8NLqAIYow) 对您了解夜莺各项关键特性会有帮助（PPT链接在文末）
|
||||
- 👉 [文档中心](https://flashcat.cloud/docs/) 为了更快的访问速度，站点托管在 [FlashcatCloud](https://flashcat.cloud)
|
||||
- ❤️ [报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml) 写清楚问题描述、复现步骤、截图等信息，更容易得到答案
|
||||
- 💡 前后端代码分离，前端代码仓库：[https://github.com/n9e/fe](https://github.com/n9e/fe)
|
||||
- 🎯 关注[这个公众号](https://gitlink.org.cn/UlricQin)了解更多夜莺动态和知识
|
||||
- 🌟 加我微信：`picobyte`（我已关闭好友验证）拉入微信群，备注：`夜莺互助群`，如果已经把夜莺上到生产环境，可联系我拉入资深监控用户群
|
||||
|
||||
## 🔑 Key Features
|
||||
|
||||
|
||||
## 关键特性简介
|
||||
|
||||
- Nightingale supports alerting rules, mute rules, subscription rules, and notification rules. It natively supports 20 types of notification media and allows customization of message templates.
|
||||
- It supports event pipelines for Pipeline processing of alarms, facilitating automated integration with in-house systems. For example, it can append metadata to alarms or perform relabeling on events.
|
||||
- It introduces the concept of business groups and a permission system to manage various rules in a categorized manner.
|
||||
- Many databases and middleware come with built-in alert rules that can be directly imported and used. It also supports direct import of Prometheus alerting rules.
|
||||
- It supports alerting self-healing, which automatically triggers a script to execute predefined logic after an alarm is generated—such as cleaning up disk space or capturing the current system state.
|
||||
|
||||
|
||||
|
||||
- 夜莺支持告警规则、屏蔽规则、订阅规则、通知规则，内置支持 20 种通知媒介，支持消息模板自定义
|
||||
- 支持事件管道，对告警事件做 Pipeline 处理，方便和自有系统做自动化整合，比如给告警事件附加一些元信息，对事件做 relabel
|
||||
- 支持业务组概念，引入权限体系，分门别类管理各类规则
|
||||
- 很多数据库、中间件内置了告警规则，可以直接导入使用，也可以直接导入 Prometheus 的告警规则
|
||||
- 支持告警自愈，即告警之后自动触发一个脚本执行一些预定义的逻辑，比如清理一下磁盘、抓一下现场等
|
||||
|
||||
- Nightingale archives historical alarms and supports multi-dimensional query and statistics.
|
||||
- It supports flexible aggregation grouping, allowing a clear view of the distribution of alarms across the company.
|
||||
|
||||
|
||||
|
||||
- 夜莺存档了历史告警事件，支持多维度的查询和统计
|
||||
- 支持灵活的聚合分组，一目了然看到公司的告警事件分布情况
|
||||
|
||||
- Nightingale has built-in metric descriptions, dashboards, and alerting rules for common operating systems, middleware, and databases, which are contributed by the community with varying quality.
|
||||
- It directly receives data via multiple protocols such as Remote Write, OpenTSDB, Datadog, and Falcon, integrates with various Agents.
|
||||
- It supports data sources like Prometheus, ElasticSearch, Loki, ClickHouse, MySQL, Postgres, allowing alerting based on data from these sources.
|
||||
- Nightingale can be easily embedded into internal enterprise systems (e.g. Grafana, CMDB), and even supports configuring menu visibility for these embedded systems.
|
||||
|
||||
|
||||
|
||||
- 夜莺内置常用操作系统、中间件、数据库的的指标说明、仪表盘、告警规则，不过都是社区贡献的，整体也是参差不齐
|
||||
- 夜莺直接接收 Remote Write、OpenTSDB、Datadog、Falcon 等多种协议的数据，故而可以和各类 Agent 对接
|
||||
- 夜莺支持 Prometheus、ElasticSearch、Loki、TDEngine 等多种数据源，可以对其中的数据做告警
|
||||
- 夜莺可以很方便内嵌企业内部系统，比如 Grafana、CMDB 等，甚至可以配置这些内嵌系统的菜单可见性
|
||||
|
||||
- Nightingale supports dashboard functionality, including common chart types, and comes with pre-built dashboards. The image above is a screenshot of one of these dashboards.
|
||||
- If you are already accustomed to Grafana, it is recommended to continue using Grafana for visualization, as Grafana has deeper expertise in this area.
|
||||
- For machine-related monitoring data collected by Categraf, it is advisable to use Nightingale's built-in dashboards for viewing. This is because Categraf's metric naming follows Telegraf's convention, which differs from that of Node Exporter.
|
||||
- Due to Nightingale's concept of business groups (where machines can belong to different groups), there may be scenarios where you only want to view machines within the current business group on the dashboard. Thus, Nightingale's dashboards can be linked with business groups for interactive filtering.
|
||||
|
||||
## 🌟 Stargazers over time
|
||||
|
||||
|
||||
- 夜莺支持仪表盘功能，支持常见的图表类型，也内置了一些仪表盘，上图是其中一个仪表盘的截图。
|
||||
- 如果你已经习惯了 Grafana，建议仍然使用 Grafana 看图。Grafana 在看图方面道行更深。
|
||||
- 机器相关的监控数据，如果是 Categraf 采集的，建议使用夜莺自带的仪表盘查看，因为 Categraf 的指标命名 Follow 的是 Telegraf 的命名方式，和 Node Exporter 不同
|
||||
- 因为夜莺有个业务组的概念，机器可以归属不同的业务组，有时在仪表盘里只想查看当前所属业务组的机器，所以夜莺的仪表盘可以和业务组联动
|
||||
|
||||
## 广受关注
|
||||
[](https://star-history.com/#ccfos/nightingale&Date)
|
||||
|
||||
## 🔥 Users
|
||||
## 感谢众多企业的信赖
|
||||
|
||||
|
||||
|
||||
|
||||
## 🤝 Community Co-Building
|
||||
|
||||
- ❇️ Please read the [Nightingale Open Source Project and Community Governance Draft](./doc/community-governance.md). We sincerely welcome every user, developer, company, and organization to use Nightingale, actively report bugs, submit feature requests, share best practices, and help build a professional and active open-source community.
|
||||
- ❤️ Nightingale Contributors
|
||||
## 社区共建
|
||||
- ❇️ 请阅读浏览[夜莺开源项目和社区治理架构草案](./doc/community-governance.md)，真诚欢迎每一位用户、开发者、公司以及组织，使用夜莺监控、积极反馈 Bug、提交功能需求、分享最佳实践，共建专业、活跃的夜莺开源社区。
|
||||
- ❤️ 夜莺贡献者
|
||||
<a href="https://github.com/ccfos/nightingale/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=ccfos/nightingale" />
|
||||
</a>
|
||||
|
||||
## 📜 License
|
||||
## License
|
||||
- [Apache License V2.0](https://github.com/didi/nightingale/blob/main/LICENSE)
|
||||
|
||||
113
README_en.md
Normal file
113
README_en.md
Normal file
@@ -0,0 +1,113 @@
|
||||
<p align="center">
|
||||
<a href="https://github.com/ccfos/nightingale">
|
||||
<img src="doc/img/Nightingale_L_V.png" alt="nightingale - cloud native monitoring" width="100" /></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<b>Open-source Alert Management Expert, an Integrated Observability Platform</b>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://flashcat.cloud/docs/">
|
||||
<img alt="Docs" src="https://img.shields.io/badge/docs-get%20started-brightgreen"/></a>
|
||||
<a href="https://hub.docker.com/u/flashcatcloud">
|
||||
<img alt="Docker pulls" src="https://img.shields.io/docker/pulls/flashcatcloud/nightingale"/></a>
|
||||
<a href="https://github.com/ccfos/nightingale/graphs/contributors">
|
||||
<img alt="GitHub contributors" src="https://img.shields.io/github/contributors-anon/ccfos/nightingale"/></a>
|
||||
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/ccfos/nightingale">
|
||||
<img alt="GitHub forks" src="https://img.shields.io/github/forks/ccfos/nightingale">
|
||||
<br/><img alt="GitHub Repo issues" src="https://img.shields.io/github/issues/ccfos/nightingale">
|
||||
<img alt="GitHub Repo issues closed" src="https://img.shields.io/github/issues-closed/ccfos/nightingale">
|
||||
<img alt="GitHub latest release" src="https://img.shields.io/github/v/release/ccfos/nightingale"/>
|
||||
<img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-blue"/>
|
||||
<a href="https://n9e-talk.slack.com/">
|
||||
<img alt="GitHub contributors" src="https://img.shields.io/badge/join%20slack-%23n9e-brightgreen.svg"/></a>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
[English](./README_en.md) | [中文](./README.md)
|
||||
|
||||
## What is Nightingale
|
||||
|
||||
Nightingale is an open-source project focused on alerting. Similar to Grafana's data source integration approach, Nightingale also connects with various existing data sources. However, while Grafana focuses on visualization, Nightingale focuses on alerting engines.
|
||||
|
||||
Originally developed and open-sourced by Didi, Nightingale was donated to the China Computer Federation Open Source Development Committee (CCF ODC) on May 11, 2022, becoming the first open-source project accepted by the CCF ODC after its establishment.
|
||||
|
||||
|
||||
## Quick Start
|
||||
|
||||
- 👉 [Documentation](https://flashcat.cloud/docs/) | [Download](https://flashcat.cloud/download/nightingale/)
|
||||
- ❤️ [Report a Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml)
|
||||
- ℹ️ For faster access, the above documentation and download sites are hosted on [FlashcatCloud](https://flashcat.cloud).
|
||||
|
||||
## Features
|
||||
|
||||
- **Integration with Multiple Time-Series Databases:** Supports integration with various time-series databases such as Prometheus, VictoriaMetrics, Thanos, Mimir, M3DB, and TDengine, enabling unified alert management.
|
||||
- **Advanced Alerting Capabilities:** Comes with built-in support for multiple alerting rules, extensible to common notification channels. It also supports alert suppression, silencing, subscription, self-healing, and alert event management.
|
||||
- **High-Performance Visualization Engine:** Offers various chart styles with numerous built-in dashboard templates and the ability to import Grafana templates. Ready to use with a business-friendly open-source license.
|
||||
- **Support for Common Collectors:** Compatible with [Categraf](https://flashcat.cloud/product/categraf), Telegraf, Grafana-agent, Datadog-agent, and various exporters as collectors—there's no data that can't be monitored.
|
||||
- **Seamless Integration with [Flashduty](https://flashcat.cloud/product/flashcat-duty/):** Enables alert aggregation, acknowledgment, escalation, scheduling, and IM integration, ensuring no alerts are missed, reducing unnecessary interruptions, and enhancing efficient collaboration.
|
||||
|
||||
|
||||
## Screenshots
|
||||
|
||||
You can switch languages and themes in the top right corner. We now support English, Simplified Chinese, and Traditional Chinese.
|
||||
|
||||
|
||||
|
||||
### Instant Query
|
||||
|
||||
Similar to the built-in query analysis page in Prometheus, Nightingale offers an ad-hoc query feature with UI enhancements. It also provides built-in PromQL metrics, allowing users unfamiliar with PromQL to quickly perform queries.
|
||||
|
||||
|
||||
|
||||
### Metric View
|
||||
|
||||
Alternatively, you can use the Metric View to access data. With this feature, Instant Query becomes less necessary, as it caters more to advanced users. Regular users can easily perform queries using the Metric View.
|
||||
|
||||
|
||||
|
||||
### Built-in Dashboards
|
||||
|
||||
Nightingale includes commonly used dashboards that can be imported and used directly. You can also import Grafana dashboards, although compatibility is limited to basic Grafana charts. If you’re accustomed to Grafana, it’s recommended to continue using it for visualization, with Nightingale serving as an alerting engine.
|
||||
|
||||
|
||||
|
||||
### Built-in Alert Rules
|
||||
|
||||
In addition to the built-in dashboards, Nightingale also comes with numerous alert rules that are ready to use out of the box.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||
In most community scenarios, Nightingale is primarily used as an alert engine, integrating with multiple time-series databases to unify alert rule management. Grafana remains the preferred tool for visualization. As an alert engine, the product architecture of Nightingale is as follows:
|
||||
|
||||
|
||||
|
||||
For certain edge data centers with poor network connectivity to the central Nightingale server, we offer a distributed deployment mode for the alert engine. In this mode, even if the network is disconnected, the alerting functionality remains unaffected.
|
||||
|
||||
|
||||
|
||||
|
||||
## Communication Channels
|
||||
|
||||
- **Report Bugs:** It is highly recommended to submit issues via the [Nightingale GitHub Issue tracker](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=kind%2Fbug&projects=&template=bug_report.yml).
|
||||
- **Documentation:** For more information, we recommend thoroughly browsing the [Nightingale Documentation Site](https://flashcat.cloud/docs/content/flashcat-monitor/nightingale-v7/introduction/).
|
||||
|
||||
## Stargazers over time
|
||||
|
||||
[](https://star-history.com/#ccfos/nightingale&Date)
|
||||
|
||||
## Community Co-Building
|
||||
|
||||
- ❇️ Please read the [Nightingale Open Source Project and Community Governance Draft](./doc/community-governance.md). We sincerely welcome every user, developer, company, and organization to use Nightingale, actively report bugs, submit feature requests, share best practices, and help build a professional and active open-source community.
|
||||
- ❤️ Nightingale Contributors
|
||||
<a href="https://github.com/ccfos/nightingale/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=ccfos/nightingale" />
|
||||
</a>
|
||||
|
||||
## License
|
||||
- [Apache License V2.0](https://github.com/didi/nightingale/blob/main/LICENSE)
|
||||
120
README_zh.md
120
README_zh.md
@@ -1,120 +0,0 @@
|
||||
<p align="center">
|
||||
<a href="https://github.com/ccfos/nightingale">
|
||||
<img src="doc/img/Nightingale_L_V.png" alt="nightingale - cloud native monitoring" width="100" /></a>
|
||||
</p>
|
||||
<p align="center">
|
||||
<b>开源告警管理专家</b>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://flashcat.cloud/docs/">
|
||||
<img alt="Docs" src="https://img.shields.io/badge/docs-get%20started-brightgreen"/></a>
|
||||
<a href="https://hub.docker.com/u/flashcatcloud">
|
||||
<img alt="Docker pulls" src="https://img.shields.io/docker/pulls/flashcatcloud/nightingale"/></a>
|
||||
<a href="https://github.com/ccfos/nightingale/graphs/contributors">
|
||||
<img alt="GitHub contributors" src="https://img.shields.io/github/contributors-anon/ccfos/nightingale"/></a>
|
||||
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/ccfos/nightingale">
|
||||
<img alt="GitHub forks" src="https://img.shields.io/github/forks/ccfos/nightingale">
|
||||
<br/><img alt="GitHub Repo issues" src="https://img.shields.io/github/issues/ccfos/nightingale">
|
||||
<img alt="GitHub Repo issues closed" src="https://img.shields.io/github/issues-closed/ccfos/nightingale">
|
||||
<img alt="GitHub latest release" src="https://img.shields.io/github/v/release/ccfos/nightingale"/>
|
||||
<img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-blue"/>
|
||||
<a href="https://n9e-talk.slack.com/">
|
||||
<img alt="GitHub contributors" src="https://img.shields.io/badge/join%20slack-%23n9e-brightgreen.svg"/></a>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
[English](./README.md) | [中文](./README_zh.md)
|
||||
|
||||
## 夜莺是什么
|
||||
|
||||
夜莺监控（Nightingale）是一款侧重告警的监控类开源项目。类似 Grafana 的数据源集成方式，夜莺也是对接多种既有的数据源，不过 Grafana 侧重在可视化，夜莺是侧重在告警引擎、告警事件的处理和分发。
|
||||
|
||||
> 夜莺监控项目，最初由滴滴开发和开源，并于 2022 年 5 月 11 日，捐赠予中国计算机学会开源发展委员会（CCF ODC），为 CCF ODC 成立后接受捐赠的第一个开源项目。
|
||||
|
||||
|
||||
|
||||
## 夜莺的工作逻辑
|
||||
|
||||
很多用户已经自行采集了指标、日志数据，此时就把存储库（VictoriaMetrics、ElasticSearch等）作为数据源接入夜莺，即可在夜莺里配置告警规则、通知规则，完成告警事件的生成和派发。
|
||||
|
||||
|
||||
|
||||
夜莺项目本身不提供监控数据采集能力。推荐您使用 [Categraf](https://github.com/flashcatcloud/categraf) 作为采集器，可以和夜莺丝滑对接。
|
||||
|
||||
[Categraf](https://github.com/flashcatcloud/categraf) 可以采集操作系统、网络设备、各类中间件、数据库的监控数据，通过 Remote Write 协议推送给夜莺，夜莺把监控数据转存到时序库（如 Prometheus、VictoriaMetrics 等），并提供告警和可视化能力。
|
||||
|
||||
对于个别边缘机房，如果和中心夜莺服务端网络链路不好，希望提升告警可用性，夜莺也提供边缘机房告警引擎下沉部署模式，这个模式下，即便边缘和中心端网络割裂，告警功能也不受影响。
|
||||
|
||||
|
||||
|
||||
> 上图中，机房A和中心机房的网络链路很好，所以直接由中心端的夜莺进程做告警引擎，机房B和中心机房的网络链路不好，所以在机房B部署了 `n9e-edge` 做告警引擎，对机房B的数据源做告警判定。
|
||||
|
||||
## 告警降噪、升级、协同
|
||||
|
||||
夜莺的侧重点是做告警引擎，即负责产生告警事件，并根据规则做灵活派发，内置支持 20 种通知媒介（电话、短信、邮件、钉钉、飞书、企微、Slack 等）。
|
||||
|
||||
如果您有更高级的需求，比如：
|
||||
|
||||
- 想要把公司的多套监控系统产生的事件聚拢到一个平台，统一做收敛降噪、响应处理、数据分析
|
||||
- 想要支持人员的排班，践行 On-call 文化，想要支持告警认领、升级（避免遗漏）、协同处理
|
||||
|
||||
那夜莺是不合适的，推荐您选用 [FlashDuty](https://flashcat.cloud/product/flashcat-duty/) 这样的 On-call 产品，产品简单易用，也有免费套餐。
|
||||
|
||||
|
||||
## 相关资料 & 交流渠道
|
||||
- 📚 [夜莺介绍PPT](https://mp.weixin.qq.com/s/Mkwx_46xrltSq8NLqAIYow) 对您了解夜莺各项关键特性会有帮助（PPT链接在文末）
|
||||
- 👉 [文档中心](https://flashcat.cloud/docs/) 为了更快的访问速度，站点托管在 [FlashcatCloud](https://flashcat.cloud)
|
||||
- ❤️ [报告 Bug](https://github.com/ccfos/nightingale/issues/new?assignees=&labels=&projects=&template=question.yml) 写清楚问题描述、复现步骤、截图等信息，更容易得到答案
|
||||
- 💡 前后端代码分离，前端代码仓库：[https://github.com/n9e/fe](https://github.com/n9e/fe)
|
||||
- 🎯 关注[这个公众号](https://gitlink.org.cn/UlricQin)了解更多夜莺动态和知识
|
||||
- 🌟 加我微信：`picobyte`（我已关闭好友验证）拉入微信群，备注：`夜莺互助群`，如果已经把夜莺上到生产环境，可联系我拉入资深监控用户群
|
||||
|
||||
|
||||
## 关键特性简介
|
||||
|
||||
|
||||
|
||||
- 夜莺支持告警规则、屏蔽规则、订阅规则、通知规则，内置支持 20 种通知媒介，支持消息模板自定义
|
||||
- 支持事件管道，对告警事件做 Pipeline 处理，方便和自有系统做自动化整合，比如给告警事件附加一些元信息，对事件做 relabel
|
||||
- 支持业务组概念，引入权限体系，分门别类管理各类规则
|
||||
- 很多数据库、中间件内置了告警规则，可以直接导入使用，也可以直接导入 Prometheus 的告警规则
|
||||
- 支持告警自愈，即告警之后自动触发一个脚本执行一些预定义的逻辑，比如清理一下磁盘、抓一下现场等
|
||||
|
||||
|
||||
|
||||
- 夜莺存档了历史告警事件，支持多维度的查询和统计
|
||||
- 支持灵活的聚合分组，一目了然看到公司的告警事件分布情况
|
||||
|
||||
|
||||
|
||||
- 夜莺内置常用操作系统、中间件、数据库的的指标说明、仪表盘、告警规则，不过都是社区贡献的，整体也是参差不齐
|
||||
- 夜莺直接接收 Remote Write、OpenTSDB、Datadog、Falcon 等多种协议的数据，故而可以和各类 Agent 对接
|
||||
- 夜莺支持 Prometheus、ElasticSearch、Loki、TDEngine 等多种数据源，可以对其中的数据做告警
|
||||
- 夜莺可以很方便内嵌企业内部系统，比如 Grafana、CMDB 等，甚至可以配置这些内嵌系统的菜单可见性
|
||||
|
||||
|
||||
|
||||
|
||||
- 夜莺支持仪表盘功能，支持常见的图表类型，也内置了一些仪表盘，上图是其中一个仪表盘的截图。
|
||||
- 如果你已经习惯了 Grafana，建议仍然使用 Grafana 看图。Grafana 在看图方面道行更深。
|
||||
- 机器相关的监控数据，如果是 Categraf 采集的，建议使用夜莺自带的仪表盘查看，因为 Categraf 的指标命名 Follow 的是 Telegraf 的命名方式，和 Node Exporter 不同
|
||||
- 因为夜莺有个业务组的概念，机器可以归属不同的业务组，有时在仪表盘里只想查看当前所属业务组的机器，所以夜莺的仪表盘可以和业务组联动
|
||||
|
||||
## 广受关注
|
||||
[](https://star-history.com/#ccfos/nightingale&Date)
|
||||
|
||||
## 感谢众多企业的信赖
|
||||
|
||||
|
||||
|
||||
## 社区共建
|
||||
- ❇️ 请阅读浏览[夜莺开源项目和社区治理架构草案](./doc/community-governance.md)，真诚欢迎每一位用户、开发者、公司以及组织，使用夜莺监控、积极反馈 Bug、提交功能需求、分享最佳实践，共建专业、活跃的夜莺开源社区。
|
||||
- ❤️ 夜莺贡献者
|
||||
<a href="https://github.com/ccfos/nightingale/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=ccfos/nightingale" />
|
||||
</a>
|
||||
|
||||
## License
|
||||
- [Apache License V2.0](https://github.com/didi/nightingale/blob/main/LICENSE)
|
||||
@@ -82,10 +82,6 @@ func NewDispatch(alertRuleCache *memsto.AlertRuleCacheType, userCache *memsto.Us
|
||||
}
|
||||
|
||||
pipeline.Init()
|
||||
|
||||
// 设置通知记录回调函数
|
||||
notifyChannelCache.SetNotifyRecordFunc(sender.NotifyRecord)
|
||||
|
||||
return notify
|
||||
}
|
||||
|
||||
@@ -451,40 +447,41 @@ func (e *Dispatch) sendV2(events []*models.AlertCurEvent, notifyRuleId int64, no
|
||||
}
|
||||
|
||||
for i := range flashDutyChannelIDs {
|
||||
start := time.Now()
|
||||
respBody, err := notifyChannel.SendFlashDuty(events, flashDutyChannelIDs[i], e.notifyChannelCache.GetHttpClient(notifyChannel.ID))
|
||||
respBody = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), respBody)
|
||||
logger.Infof("notify_id: %d, channel_name: %v, event:%+v, IntegrationUrl: %v dutychannel_id: %v, respBody: %v, err: %v", notifyRuleId, notifyChannel.Name, events[0], notifyChannel.RequestConfig.FlashDutyRequestConfig.IntegrationUrl, flashDutyChannelIDs[i], respBody, err)
|
||||
sender.NotifyRecord(e.ctx, events, notifyRuleId, notifyChannel.Name, strconv.FormatInt(flashDutyChannelIDs[i], 10), respBody, err)
|
||||
}
|
||||
|
||||
return
|
||||
case "http":
|
||||
// 使用队列模式处理 http 通知
|
||||
// 创建通知任务
|
||||
task := &memsto.NotifyTask{
|
||||
Events: events,
|
||||
NotifyRuleId: notifyRuleId,
|
||||
NotifyChannel: notifyChannel,
|
||||
TplContent: tplContent,
|
||||
CustomParams: customParams,
|
||||
Sendtos: sendtos,
|
||||
if e.notifyChannelCache.HttpConcurrencyAdd(notifyChannel.ID) {
|
||||
defer e.notifyChannelCache.HttpConcurrencyDone(notifyChannel.ID)
|
||||
}
|
||||
if notifyChannel.RequestConfig == nil {
|
||||
logger.Warningf("notify_id: %d, channel_name: %v, event:%+v, request config not found", notifyRuleId, notifyChannel.Name, events[0])
|
||||
}
|
||||
|
||||
// 将任务加入队列
|
||||
success := e.notifyChannelCache.EnqueueNotifyTask(task)
|
||||
if !success {
|
||||
logger.Errorf("failed to enqueue notify task for channel %d, notify_id: %d", notifyChannel.ID, notifyRuleId)
|
||||
// 如果入队失败，记录错误通知
|
||||
sender.NotifyRecord(e.ctx, events, notifyRuleId, notifyChannel.Name, getSendTarget(customParams, sendtos), "", errors.New("failed to enqueue notify task, queue is full"))
|
||||
if notifyChannel.RequestConfig.HTTPRequestConfig == nil {
|
||||
logger.Warningf("notify_id: %d, channel_name: %v, event:%+v, http request config not found", notifyRuleId, notifyChannel.Name, events[0])
|
||||
}
|
||||
|
||||
if NeedBatchContacts(notifyChannel.RequestConfig.HTTPRequestConfig) || len(sendtos) == 0 {
|
||||
resp, err := notifyChannel.SendHTTP(events, tplContent, customParams, sendtos, e.notifyChannelCache.GetHttpClient(notifyChannel.ID))
|
||||
logger.Infof("notify_id: %d, channel_name: %v, event:%+v, tplContent:%s, customParams:%v, userInfo:%+v, respBody: %v, err: %v", notifyRuleId, notifyChannel.Name, events[0], tplContent, customParams, sendtos, resp, err)
|
||||
|
||||
sender.NotifyRecord(e.ctx, events, notifyRuleId, notifyChannel.Name, getSendTarget(customParams, sendtos), resp, err)
|
||||
} else {
|
||||
for i := range sendtos {
|
||||
resp, err := notifyChannel.SendHTTP(events, tplContent, customParams, []string{sendtos[i]}, e.notifyChannelCache.GetHttpClient(notifyChannel.ID))
|
||||
logger.Infof("notify_id: %d, channel_name: %v, event:%+v, tplContent:%s, customParams:%v, userInfo:%+v, respBody: %v, err: %v", notifyRuleId, notifyChannel.Name, events[0], tplContent, customParams, sendtos[i], resp, err)
|
||||
sender.NotifyRecord(e.ctx, events, notifyRuleId, notifyChannel.Name, getSendTarget(customParams, []string{sendtos[i]}), resp, err)
|
||||
}
|
||||
}
|
||||
|
||||
case "smtp":
|
||||
notifyChannel.SendEmail(notifyRuleId, events, tplContent, sendtos, e.notifyChannelCache.GetSmtpClient(notifyChannel.ID))
|
||||
|
||||
case "script":
|
||||
start := time.Now()
|
||||
target, res, err := notifyChannel.SendScript(events, tplContent, customParams, sendtos)
|
||||
res = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), res)
|
||||
logger.Infof("notify_id: %d, channel_name: %v, event:%+v, tplContent:%s, customParams:%v, target:%s, res:%s, err:%v", notifyRuleId, notifyChannel.Name, events[0], tplContent, customParams, target, res, err)
|
||||
sender.NotifyRecord(e.ctx, events, notifyRuleId, notifyChannel.Name, target, res, err)
|
||||
default:
|
||||
|
||||
@@ -144,24 +144,14 @@ func (arw *AlertRuleWorker) Start() {
|
||||
}
|
||||
|
||||
func (arw *AlertRuleWorker) Eval() {
|
||||
begin := time.Now()
|
||||
var message string
|
||||
|
||||
defer func() {
|
||||
if len(message) == 0 {
|
||||
logger.Infof("rule_eval:%s finished, duration:%v", arw.Key(), time.Since(begin))
|
||||
} else {
|
||||
logger.Infof("rule_eval:%s finished, duration:%v, message:%s", arw.Key(), time.Since(begin), message)
|
||||
}
|
||||
}()
|
||||
|
||||
logger.Infof("eval:%s started", arw.Key())
|
||||
if arw.Processor.PromEvalInterval == 0 {
|
||||
arw.Processor.PromEvalInterval = getPromEvalInterval(arw.Processor.ScheduleEntry.Schedule)
|
||||
}
|
||||
|
||||
cachedRule := arw.Rule
|
||||
if cachedRule == nil {
|
||||
message = "rule not found"
|
||||
// logger.Errorf("rule_eval:%s Rule not found", arw.Key())
|
||||
return
|
||||
}
|
||||
arw.Processor.Stats.CounterRuleEval.WithLabelValues().Inc()
|
||||
@@ -187,12 +177,11 @@ func (arw *AlertRuleWorker) Eval() {
|
||||
|
||||
if err != nil {
|
||||
logger.Errorf("rule_eval:%s get anomaly point err:%s", arw.Key(), err.Error())
|
||||
message = "failed to get anomaly points"
|
||||
return
|
||||
}
|
||||
|
||||
if arw.Processor == nil {
|
||||
message = "processor is nil"
|
||||
logger.Warningf("rule_eval:%s Processor is nil", arw.Key())
|
||||
return
|
||||
}
|
||||
|
||||
@@ -234,7 +223,7 @@ func (arw *AlertRuleWorker) Eval() {
|
||||
}
|
||||
|
||||
func (arw *AlertRuleWorker) Stop() {
|
||||
logger.Infof("rule_eval:%s stopped", arw.Key())
|
||||
logger.Infof("rule_eval %s stopped", arw.Key())
|
||||
close(arw.Quit)
|
||||
c := arw.Scheduler.Stop()
|
||||
<-c.Done()
|
||||
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
"github.com/ccfos/nightingale/v6/memsto"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
|
||||
"github.com/pkg/errors"
|
||||
"github.com/toolkits/pkg/logger"
|
||||
)
|
||||
|
||||
@@ -136,8 +135,7 @@ func EventMuteStrategy(event *models.AlertCurEvent, alertMuteCache *memsto.Alert
|
||||
}
|
||||
|
||||
for i := 0; i < len(mutes); i++ {
|
||||
matched, _ := MatchMute(event, mutes[i])
|
||||
if matched {
|
||||
if MatchMute(event, mutes[i]) {
|
||||
return true, mutes[i].Id
|
||||
}
|
||||
}
|
||||
@@ -146,9 +144,9 @@ func EventMuteStrategy(event *models.AlertCurEvent, alertMuteCache *memsto.Alert
|
||||
}
|
||||
|
||||
// MatchMute 如果传入了clock这个可选参数，就表示使用这个clock表示的时间，否则就从event的字段中取TriggerTime
|
||||
func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int64) (bool, error) {
|
||||
func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int64) bool {
|
||||
if mute.Disabled == 1 {
|
||||
return false, errors.New("mute is disabled")
|
||||
return false
|
||||
}
|
||||
|
||||
// 如果不是全局的，判断 匹配的 datasource id
|
||||
@@ -160,13 +158,13 @@ func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int
|
||||
|
||||
// 判断 event.datasourceId 是否包含在 idm 中
|
||||
if _, has := idm[event.DatasourceId]; !has {
|
||||
return false, errors.New("datasource id not match")
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
if mute.MuteTimeType == models.TimeRange {
|
||||
if !mute.IsWithinTimeRange(event.TriggerTime) {
|
||||
return false, errors.New("event trigger time not within mute time range")
|
||||
return false
|
||||
}
|
||||
} else if mute.MuteTimeType == models.Periodic {
|
||||
ts := event.TriggerTime
|
||||
@@ -175,11 +173,11 @@ func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int
|
||||
}
|
||||
|
||||
if !mute.IsWithinPeriodicMute(ts) {
|
||||
return false, errors.New("event trigger time not within periodic mute range")
|
||||
return false
|
||||
}
|
||||
} else {
|
||||
logger.Warningf("mute time type invalid, %d", mute.MuteTimeType)
|
||||
return false, errors.New("mute time type invalid")
|
||||
return false
|
||||
}
|
||||
|
||||
var matchSeverity bool
|
||||
@@ -195,14 +193,12 @@ func MatchMute(event *models.AlertCurEvent, mute *models.AlertMute, clock ...int
|
||||
}
|
||||
|
||||
if !matchSeverity {
|
||||
return false, errors.New("event severity not match mute severity")
|
||||
return false
|
||||
}
|
||||
|
||||
if mute.ITags == nil || len(mute.ITags) == 0 {
|
||||
return true, nil
|
||||
return true
|
||||
}
|
||||
if !common.MatchTags(event.TagsMap, mute.ITags) {
|
||||
return false, errors.New("event tags not match mute tags")
|
||||
}
|
||||
return true, nil
|
||||
|
||||
return common.MatchTags(event.TagsMap, mute.ITags)
|
||||
}
|
||||
|
||||
@@ -53,7 +53,7 @@ func (c *EventDropConfig) Process(ctx *ctx.Context, event *models.AlertCurEvent)
|
||||
logger.Infof("processor eventdrop result: %v", result)
|
||||
if result == "true" {
|
||||
logger.Infof("processor eventdrop drop event: %v", event)
|
||||
return nil, "drop event success", nil
|
||||
return event, "drop event success", nil
|
||||
}
|
||||
|
||||
return event, "drop event failed", nil
|
||||
|
||||
@@ -467,18 +467,16 @@ func (p *Processor) fireEvent(event *models.AlertCurEvent) {
|
||||
return
|
||||
}
|
||||
|
||||
message := "unknown"
|
||||
defer func() {
|
||||
logger.Infof("rule_eval:%s event-hash-%s %s", p.Key(), event.Hash, message)
|
||||
}()
|
||||
|
||||
logger.Debugf("rule_eval:%s event:%+v fire", p.Key(), event)
|
||||
if fired, has := p.fires.Get(event.Hash); has {
|
||||
p.fires.UpdateLastEvalTime(event.Hash, event.LastEvalTime)
|
||||
event.FirstTriggerTime = fired.FirstTriggerTime
|
||||
p.HandleFireEventHook(event)
|
||||
|
||||
if cachedRule.NotifyRepeatStep == 0 {
|
||||
message = "stalled, rule.notify_repeat_step is 0, no need to repeat notify"
|
||||
logger.Debugf("rule_eval:%s event:%+v repeat is zero nothing to do", p.Key(), event)
|
||||
// 说明不想重复通知，那就直接返回了，nothing to do
|
||||
// do not need to send alert again
|
||||
return
|
||||
}
|
||||
|
||||
@@ -487,26 +485,21 @@ func (p *Processor) fireEvent(event *models.AlertCurEvent) {
|
||||
if cachedRule.NotifyMaxNumber == 0 {
|
||||
// 最大可以发送次数如果是0，表示不想限制最大发送次数，一直发即可
|
||||
event.NotifyCurNumber = fired.NotifyCurNumber + 1
|
||||
message = fmt.Sprintf("fired, notify_repeat_step_matched(%d >= %d + %d * 60) notify_max_number_ignore(#%d / %d)", event.LastEvalTime, fired.LastSentTime, cachedRule.NotifyRepeatStep, event.NotifyCurNumber, cachedRule.NotifyMaxNumber)
|
||||
p.pushEventToQueue(event)
|
||||
} else {
|
||||
// 有最大发送次数的限制，就要看已经发了几次了，是否达到了最大发送次数
|
||||
if fired.NotifyCurNumber >= cachedRule.NotifyMaxNumber {
|
||||
message = fmt.Sprintf("stalled, notify_repeat_step_matched(%d >= %d + %d * 60) notify_max_number_not_matched(#%d / %d)", event.LastEvalTime, fired.LastSentTime, cachedRule.NotifyRepeatStep, fired.NotifyCurNumber, cachedRule.NotifyMaxNumber)
|
||||
logger.Debugf("rule_eval:%s event:%+v reach max number", p.Key(), event)
|
||||
return
|
||||
} else {
|
||||
event.NotifyCurNumber = fired.NotifyCurNumber + 1
|
||||
message = fmt.Sprintf("fired, notify_repeat_step_matched(%d >= %d + %d * 60) notify_max_number_matched(#%d / %d)", event.LastEvalTime, fired.LastSentTime, cachedRule.NotifyRepeatStep, event.NotifyCurNumber, cachedRule.NotifyMaxNumber)
|
||||
p.pushEventToQueue(event)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
message = fmt.Sprintf("stalled, notify_repeat_step_not_matched(%d < %d + %d * 60)", event.LastEvalTime, fired.LastSentTime, cachedRule.NotifyRepeatStep)
|
||||
}
|
||||
} else {
|
||||
event.NotifyCurNumber = 1
|
||||
event.FirstTriggerTime = event.TriggerTime
|
||||
message = fmt.Sprintf("fired, first_trigger_time: %d", event.FirstTriggerTime)
|
||||
p.HandleFireEventHook(event)
|
||||
p.pushEventToQueue(event)
|
||||
}
|
||||
@@ -584,9 +577,7 @@ func (p *Processor) fillTags(anomalyPoint models.AnomalyPoint) {
|
||||
}
|
||||
|
||||
// handle rule tags
|
||||
tags := p.rule.AppendTagsJSON
|
||||
tags = append(tags, "rulename="+p.rule.Name)
|
||||
for _, tag := range tags {
|
||||
for _, tag := range p.rule.AppendTagsJSON {
|
||||
arr := strings.SplitN(tag, "=", 2)
|
||||
|
||||
var defs = []string{
|
||||
@@ -612,6 +603,8 @@ func (p *Processor) fillTags(anomalyPoint models.AnomalyPoint) {
|
||||
|
||||
tagsMap[arr[0]] = body.String()
|
||||
}
|
||||
|
||||
tagsMap["rulename"] = p.rule.Name
|
||||
p.tagsMap = tagsMap
|
||||
|
||||
// handle tagsArr
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package sender
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"html/template"
|
||||
"net/url"
|
||||
"strings"
|
||||
@@ -135,9 +134,7 @@ func (c *DefaultCallBacker) CallBack(ctx CallBackContext) {
|
||||
|
||||
func doSendAndRecord(ctx *ctx.Context, url, token string, body interface{}, channel string,
|
||||
stats *astats.Stats, events []*models.AlertCurEvent) {
|
||||
start := time.Now()
|
||||
res, err := doSend(url, body, channel, stats)
|
||||
res = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), res)
|
||||
NotifyRecord(ctx, events, 0, channel, token, res, err)
|
||||
}
|
||||
|
||||
@@ -169,9 +166,7 @@ func NotifyRecord(ctx *ctx.Context, evts []*models.AlertCurEvent, notifyRuleID i
|
||||
func doSend(url string, body interface{}, channel string, stats *astats.Stats) (string, error) {
|
||||
stats.AlertNotifyTotal.WithLabelValues(channel).Inc()
|
||||
|
||||
start := time.Now()
|
||||
res, code, err := poster.PostJSON(url, time.Second*5, body, 3)
|
||||
res = []byte(fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), res))
|
||||
if err != nil {
|
||||
logger.Errorf("%s_sender: result=fail url=%s code=%d error=%v req:%v response=%s", channel, url, code, err, body, string(res))
|
||||
stats.AlertNotifyErrorTotal.WithLabelValues(channel).Inc()
|
||||
|
||||
@@ -79,7 +79,6 @@ func alertingCallScript(ctx *ctx.Context, stdinBytes []byte, notifyScript models
|
||||
cmd.Stdout = &buf
|
||||
cmd.Stderr = &buf
|
||||
|
||||
start := time.Now()
|
||||
err := startCmd(cmd)
|
||||
if err != nil {
|
||||
logger.Errorf("event_script_notify_fail: run cmd err: %v", err)
|
||||
@@ -89,7 +88,6 @@ func alertingCallScript(ctx *ctx.Context, stdinBytes []byte, notifyScript models
|
||||
err, isTimeout := sys.WrapTimeout(cmd, time.Duration(config.Timeout)*time.Second)
|
||||
|
||||
res := buf.String()
|
||||
res = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), res)
|
||||
|
||||
// 截断超出长度的输出
|
||||
if len(res) > 512 {
|
||||
|
||||
@@ -99,9 +99,7 @@ func SingleSendWebhooks(ctx *ctx.Context, webhooks map[string]*models.Webhook, e
|
||||
for _, conf := range webhooks {
|
||||
retryCount := 0
|
||||
for retryCount < 3 {
|
||||
start := time.Now()
|
||||
needRetry, res, err := sendWebhook(conf, event, stats)
|
||||
res = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), res)
|
||||
NotifyRecord(ctx, []*models.AlertCurEvent{event}, 0, "webhook", conf.Url, res, err)
|
||||
if !needRetry {
|
||||
break
|
||||
@@ -171,9 +169,7 @@ func StartConsumer(ctx *ctx.Context, queue *WebhookQueue, popSize int, webhook *
|
||||
|
||||
retryCount := 0
|
||||
for retryCount < webhook.RetryCount {
|
||||
start := time.Now()
|
||||
needRetry, res, err := sendWebhook(webhook, events, stats)
|
||||
res = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), res)
|
||||
go NotifyRecord(ctx, events, 0, "webhook", webhook.Url, res, err)
|
||||
if !needRetry {
|
||||
break
|
||||
|
||||
@@ -43,16 +43,4 @@ var Plugins = []Plugin{
|
||||
Type: "pgsql",
|
||||
TypeName: "PostgreSQL",
|
||||
},
|
||||
{
|
||||
Id: 8,
|
||||
Category: "logging",
|
||||
Type: "doris",
|
||||
TypeName: "Doris",
|
||||
},
|
||||
{
|
||||
Id: 9,
|
||||
Category: "logging",
|
||||
Type: "opensearch",
|
||||
TypeName: "OpenSearch",
|
||||
},
|
||||
}
|
||||
|
||||
@@ -177,7 +177,6 @@ func (rt *Router) Config(r *gin.Engine) {
|
||||
pages := r.Group(pagesPrefix)
|
||||
{
|
||||
|
||||
pages.DELETE("/datasource/series", rt.auth(), rt.admin(), rt.deleteDatasourceSeries)
|
||||
if rt.Center.AnonymousAccess.PromQuerier {
|
||||
pages.Any("/proxy/:id/*url", rt.dsProxy)
|
||||
pages.POST("/query-range-batch", rt.promBatchQueryRange)
|
||||
@@ -232,11 +231,6 @@ func (rt *Router) Config(r *gin.Engine) {
|
||||
pages.POST("/log-query", rt.QueryLog)
|
||||
}
|
||||
|
||||
// OpenSearch 专用接口
|
||||
pages.POST("/os-indices", rt.QueryOSIndices)
|
||||
pages.POST("/os-variable", rt.QueryOSVariable)
|
||||
pages.POST("/os-fields", rt.QueryOSFields)
|
||||
|
||||
pages.GET("/sql-template", rt.QuerySqlTemplate)
|
||||
pages.POST("/auth/login", rt.jwtMock(), rt.loginPost)
|
||||
pages.POST("/auth/logout", rt.jwtMock(), rt.auth(), rt.user(), rt.logoutPost)
|
||||
@@ -260,7 +254,6 @@ func (rt *Router) Config(r *gin.Engine) {
|
||||
|
||||
pages.GET("/notify-channels", rt.notifyChannelsGets)
|
||||
pages.GET("/contact-keys", rt.contactKeysGets)
|
||||
pages.GET("/install-date", rt.installDateGet)
|
||||
|
||||
pages.GET("/self/perms", rt.auth(), rt.user(), rt.permsGets)
|
||||
pages.GET("/self/profile", rt.auth(), rt.user(), rt.selfProfileGet)
|
||||
@@ -379,8 +372,6 @@ func (rt *Router) Config(r *gin.Engine) {
|
||||
pages.POST("/relabel-test", rt.auth(), rt.user(), rt.relabelTest)
|
||||
pages.POST("/busi-group/:id/alert-rules/clone", rt.auth(), rt.user(), rt.perm("/alert-rules/add"), rt.bgrw(), rt.cloneToMachine)
|
||||
pages.POST("/busi-groups/alert-rules/clones", rt.auth(), rt.user(), rt.perm("/alert-rules/add"), rt.batchAlertRuleClone)
|
||||
pages.POST("/busi-group/alert-rules/notify-tryrun", rt.auth(), rt.user(), rt.perm("/alert-rules/add"), rt.alertRuleNotifyTryRun)
|
||||
pages.POST("/busi-group/alert-rules/enable-tryrun", rt.auth(), rt.user(), rt.perm("/alert-rules/add"), rt.alertRuleEnableTryRun)
|
||||
|
||||
pages.GET("/busi-groups/recording-rules", rt.auth(), rt.user(), rt.perm("/recording-rules"), rt.recordingRuleGetsByGids)
|
||||
pages.GET("/busi-group/:id/recording-rules", rt.auth(), rt.user(), rt.perm("/recording-rules"), rt.recordingRuleGets)
|
||||
@@ -406,7 +397,6 @@ func (rt *Router) Config(r *gin.Engine) {
|
||||
pages.POST("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/add"), rt.bgrw(), rt.alertSubscribeAdd)
|
||||
pages.PUT("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/put"), rt.bgrw(), rt.alertSubscribePut)
|
||||
pages.DELETE("/busi-group/:id/alert-subscribes", rt.auth(), rt.user(), rt.perm("/alert-subscribes/del"), rt.bgrw(), rt.alertSubscribeDel)
|
||||
pages.POST("/alert-subscribe/alert-subscribes-tryrun", rt.auth(), rt.user(), rt.perm("/alert-subscribes/add"), rt.alertSubscribeTryRun)
|
||||
|
||||
pages.GET("/alert-cur-event/:eid", rt.alertCurEventGet)
|
||||
pages.GET("/alert-his-event/:eid", rt.alertHisEventGet)
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
|
||||
"gopkg.in/yaml.v2"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/alert/mute"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
"github.com/ccfos/nightingale/v6/pkg/strx"
|
||||
"github.com/ccfos/nightingale/v6/pushgw/pconf"
|
||||
@@ -19,7 +18,6 @@ import (
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/jinzhu/copier"
|
||||
"github.com/pkg/errors"
|
||||
"github.com/prometheus/prometheus/prompb"
|
||||
"github.com/toolkits/pkg/ginx"
|
||||
"github.com/toolkits/pkg/i18n"
|
||||
@@ -159,120 +157,6 @@ func (rt *Router) alertRuleAddByFE(c *gin.Context) {
|
||||
ginx.NewRender(c).Data(reterr, nil)
|
||||
}
|
||||
|
||||
type AlertRuleTryRunForm struct {
|
||||
EventId int64 `json:"event_id" binding:"required"`
|
||||
AlertRuleConfig models.AlertRule `json:"config" binding:"required"`
|
||||
}
|
||||
|
||||
func (rt *Router) alertRuleNotifyTryRun(c *gin.Context) {
|
||||
// check notify channels of old version
|
||||
var f AlertRuleTryRunForm
|
||||
ginx.BindJSON(c, &f)
|
||||
|
||||
hisEvent, err := models.AlertHisEventGetById(rt.Ctx, f.EventId)
|
||||
ginx.Dangerous(err)
|
||||
|
||||
if hisEvent == nil {
|
||||
ginx.Bomb(http.StatusNotFound, "event not found")
|
||||
}
|
||||
|
||||
curEvent := *hisEvent.ToCur()
|
||||
curEvent.SetTagsMap()
|
||||
|
||||
if f.AlertRuleConfig.NotifyVersion == 1 {
|
||||
for _, id := range f.AlertRuleConfig.NotifyRuleIds {
|
||||
notifyRule, err := models.GetNotifyRule(rt.Ctx, id)
|
||||
ginx.Dangerous(err)
|
||||
for _, notifyConfig := range notifyRule.NotifyConfigs {
|
||||
_, err = SendNotifyChannelMessage(rt.Ctx, rt.UserCache, rt.UserGroupCache, notifyConfig, []*models.AlertCurEvent{&curEvent})
|
||||
ginx.Dangerous(err)
|
||||
}
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data("notification test ok", nil)
|
||||
return
|
||||
}
|
||||
|
||||
if len(f.AlertRuleConfig.NotifyChannelsJSON) == 0 {
|
||||
ginx.Bomb(http.StatusOK, "no notify channels selected")
|
||||
}
|
||||
|
||||
if len(f.AlertRuleConfig.NotifyGroupsJSON) == 0 {
|
||||
ginx.Bomb(http.StatusOK, "no notify groups selected")
|
||||
}
|
||||
|
||||
ancs := make([]string, 0, len(curEvent.NotifyChannelsJSON))
|
||||
ugids := f.AlertRuleConfig.NotifyGroupsJSON
|
||||
ngids := make([]int64, 0)
|
||||
for i := 0; i < len(ugids); i++ {
|
||||
if gid, err := strconv.ParseInt(ugids[i], 10, 64); err == nil {
|
||||
ngids = append(ngids, gid)
|
||||
}
|
||||
}
|
||||
userGroups := rt.UserGroupCache.GetByUserGroupIds(ngids)
|
||||
uids := make([]int64, 0)
|
||||
for i := range userGroups {
|
||||
uids = append(uids, userGroups[i].UserIds...)
|
||||
}
|
||||
users := rt.UserCache.GetByUserIds(uids)
|
||||
for _, NotifyChannels := range curEvent.NotifyChannelsJSON {
|
||||
flag := true
|
||||
// ignore non-default channels
|
||||
switch NotifyChannels {
|
||||
case models.Dingtalk, models.Wecom, models.Feishu, models.Mm,
|
||||
models.Telegram, models.Email, models.FeishuCard:
|
||||
// do nothing
|
||||
default:
|
||||
continue
|
||||
}
|
||||
// default channels
|
||||
for ui := range users {
|
||||
if _, b := users[ui].ExtractToken(NotifyChannels); b {
|
||||
flag = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if flag {
|
||||
ancs = append(ancs, NotifyChannels)
|
||||
}
|
||||
}
|
||||
if len(ancs) > 0 {
|
||||
ginx.Dangerous(errors.New(fmt.Sprintf("All users are missing notify channel configurations. Please check for missing tokens (each channel should be configured with at least one user). %v", ancs)))
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data("notification test ok", nil)
|
||||
}
|
||||
|
||||
func (rt *Router) alertRuleEnableTryRun(c *gin.Context) {
|
||||
// check notify channels of old version
|
||||
var f AlertRuleTryRunForm
|
||||
ginx.BindJSON(c, &f)
|
||||
|
||||
hisEvent, err := models.AlertHisEventGetById(rt.Ctx, f.EventId)
|
||||
ginx.Dangerous(err)
|
||||
|
||||
if hisEvent == nil {
|
||||
ginx.Bomb(http.StatusNotFound, "event not found")
|
||||
}
|
||||
|
||||
curEvent := *hisEvent.ToCur()
|
||||
curEvent.SetTagsMap()
|
||||
|
||||
if f.AlertRuleConfig.Disabled == 1 {
|
||||
ginx.Bomb(http.StatusOK, "rule is disabled")
|
||||
}
|
||||
|
||||
if mute.TimeSpanMuteStrategy(&f.AlertRuleConfig, &curEvent) {
|
||||
ginx.Bomb(http.StatusOK, "event is not match for period of time")
|
||||
}
|
||||
|
||||
if mute.BgNotMatchMuteStrategy(&f.AlertRuleConfig, &curEvent, rt.TargetCache) {
|
||||
ginx.Bomb(http.StatusOK, "event target busi group not match rule busi group")
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data("event is effective", nil)
|
||||
}
|
||||
|
||||
func (rt *Router) alertRuleAddByImport(c *gin.Context) {
|
||||
username := c.MustGet("username").(string)
|
||||
|
||||
|
||||
@@ -2,17 +2,13 @@ package router
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/alert/common"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
"github.com/ccfos/nightingale/v6/pkg/strx"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/toolkits/pkg/ginx"
|
||||
"github.com/toolkits/pkg/i18n"
|
||||
)
|
||||
|
||||
// Return all, front-end search and paging
|
||||
@@ -108,148 +104,6 @@ func (rt *Router) alertSubscribeAdd(c *gin.Context) {
|
||||
ginx.NewRender(c).Message(f.Add(rt.Ctx))
|
||||
}
|
||||
|
||||
type SubscribeTryRunForm struct {
|
||||
EventId int64 `json:"event_id" binding:"required"`
|
||||
SubscribeConfig models.AlertSubscribe `json:"config" binding:"required"`
|
||||
}
|
||||
|
||||
func (rt *Router) alertSubscribeTryRun(c *gin.Context) {
|
||||
var f SubscribeTryRunForm
|
||||
ginx.BindJSON(c, &f)
|
||||
ginx.Dangerous(f.SubscribeConfig.Verify())
|
||||
|
||||
hisEvent, err := models.AlertHisEventGetById(rt.Ctx, f.EventId)
|
||||
ginx.Dangerous(err)
|
||||
|
||||
if hisEvent == nil {
|
||||
ginx.Bomb(http.StatusNotFound, "event not found")
|
||||
}
|
||||
|
||||
curEvent := *hisEvent.ToCur()
|
||||
curEvent.SetTagsMap()
|
||||
|
||||
lang := c.GetHeader("X-Language")
|
||||
|
||||
// 先判断匹配条件
|
||||
if !f.SubscribeConfig.MatchCluster(curEvent.DatasourceId) {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "event datasource not match"))
|
||||
}
|
||||
|
||||
if len(f.SubscribeConfig.RuleIds) != 0 {
|
||||
match := false
|
||||
for _, rid := range f.SubscribeConfig.RuleIds {
|
||||
if rid == curEvent.RuleId {
|
||||
match = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !match {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "event rule id not match"))
|
||||
}
|
||||
}
|
||||
|
||||
// 匹配 tag
|
||||
f.SubscribeConfig.Parse()
|
||||
if !common.MatchTags(curEvent.TagsMap, f.SubscribeConfig.ITags) {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "event tags not match"))
|
||||
}
|
||||
|
||||
// 匹配group name
|
||||
if !common.MatchGroupsName(curEvent.GroupName, f.SubscribeConfig.IBusiGroups) {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "event group name not match"))
|
||||
}
|
||||
|
||||
// 检查严重级别（Severity）匹配
|
||||
if len(f.SubscribeConfig.SeveritiesJson) != 0 {
|
||||
match := false
|
||||
for _, s := range f.SubscribeConfig.SeveritiesJson {
|
||||
if s == curEvent.Severity || s == 0 {
|
||||
match = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !match {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "event severity not match"))
|
||||
}
|
||||
}
|
||||
|
||||
// 新版本通知规则
|
||||
if f.SubscribeConfig.NotifyVersion == 1 {
|
||||
if len(f.SubscribeConfig.NotifyRuleIds) == 0 {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "no notify rules selected"))
|
||||
}
|
||||
|
||||
for _, id := range f.SubscribeConfig.NotifyRuleIds {
|
||||
notifyRule, err := models.GetNotifyRule(rt.Ctx, id)
|
||||
if err != nil {
|
||||
ginx.Bomb(http.StatusNotFound, i18n.Sprintf(lang, "subscribe notify rule not found: %v", err))
|
||||
}
|
||||
|
||||
for _, notifyConfig := range notifyRule.NotifyConfigs {
|
||||
_, err = SendNotifyChannelMessage(rt.Ctx, rt.UserCache, rt.UserGroupCache, notifyConfig, []*models.AlertCurEvent{&curEvent})
|
||||
if err != nil {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "notify rule send error: %v", err))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data(i18n.Sprintf(lang, "event match subscribe and notification test ok"), nil)
|
||||
return
|
||||
}
|
||||
|
||||
// 旧版通知方式
|
||||
f.SubscribeConfig.ModifyEvent(&curEvent)
|
||||
if len(curEvent.NotifyChannelsJSON) == 0 {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "no notify channels selected"))
|
||||
}
|
||||
|
||||
if len(curEvent.NotifyGroupsJSON) == 0 {
|
||||
ginx.Bomb(http.StatusOK, i18n.Sprintf(lang, "no notify groups selected"))
|
||||
}
|
||||
|
||||
ancs := make([]string, 0, len(curEvent.NotifyChannelsJSON))
|
||||
ugids := strings.Fields(f.SubscribeConfig.UserGroupIds)
|
||||
ngids := make([]int64, 0)
|
||||
for i := 0; i < len(ugids); i++ {
|
||||
if gid, err := strconv.ParseInt(ugids[i], 10, 64); err == nil {
|
||||
ngids = append(ngids, gid)
|
||||
}
|
||||
}
|
||||
|
||||
userGroups := rt.UserGroupCache.GetByUserGroupIds(ngids)
|
||||
uids := make([]int64, 0)
|
||||
for i := range userGroups {
|
||||
uids = append(uids, userGroups[i].UserIds...)
|
||||
}
|
||||
users := rt.UserCache.GetByUserIds(uids)
|
||||
for _, NotifyChannels := range curEvent.NotifyChannelsJSON {
|
||||
flag := true
|
||||
// ignore non-default channels
|
||||
switch NotifyChannels {
|
||||
case models.Dingtalk, models.Wecom, models.Feishu, models.Mm,
|
||||
models.Telegram, models.Email, models.FeishuCard:
|
||||
// do nothing
|
||||
default:
|
||||
continue
|
||||
}
|
||||
// default channels
|
||||
for ui := range users {
|
||||
if _, b := users[ui].ExtractToken(NotifyChannels); b {
|
||||
flag = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if flag {
|
||||
ancs = append(ancs, NotifyChannels)
|
||||
}
|
||||
}
|
||||
if len(ancs) > 0 {
|
||||
ginx.Bomb(http.StatusBadRequest, i18n.Sprintf(lang, "all users missing notify channel configurations: %v", ancs))
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data(i18n.Sprintf(lang, "event match subscribe and notify settings ok"), nil)
|
||||
}
|
||||
|
||||
func (rt *Router) alertSubscribePut(c *gin.Context) {
|
||||
var fs []models.AlertSubscribe
|
||||
ginx.BindJSON(c, &fs)
|
||||
|
||||
@@ -2,14 +2,12 @@ package router
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/datasource/opensearch"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
@@ -110,48 +108,6 @@ func (rt *Router) datasourceUpsert(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
for k, v := range req.SettingsJson {
|
||||
if strings.Contains(k, "cluster_name") {
|
||||
req.ClusterName = v.(string)
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if req.PluginType == models.OPENSEARCH {
|
||||
b, err := json.Marshal(req.SettingsJson)
|
||||
if err != nil {
|
||||
logger.Warningf("marshal settings fail: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
var os opensearch.OpenSearch
|
||||
err = json.Unmarshal(b, &os)
|
||||
if err != nil {
|
||||
logger.Warningf("unmarshal settings fail: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(os.Nodes) == 0 {
|
||||
logger.Warningf("nodes empty, %+v", req)
|
||||
return
|
||||
}
|
||||
|
||||
req.HTTPJson = models.HTTP{
|
||||
Timeout: os.Timeout,
|
||||
Url: os.Nodes[0],
|
||||
Headers: os.Headers,
|
||||
TLS: models.TLS{
|
||||
SkipTlsVerify: os.TLS.SkipTlsVerify,
|
||||
},
|
||||
}
|
||||
|
||||
req.AuthJson = models.Auth{
|
||||
BasicAuth: os.Basic.Enable,
|
||||
BasicAuthUser: os.Basic.Username,
|
||||
BasicAuthPassword: os.Basic.Password,
|
||||
}
|
||||
}
|
||||
|
||||
if req.Id == 0 {
|
||||
req.CreatedBy = username
|
||||
req.Status = "enabled"
|
||||
|
||||
@@ -158,11 +158,7 @@ func (rt *Router) tryRunEventPipeline(c *gin.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
m := map[string]interface{}{
|
||||
"event": event,
|
||||
"result": "",
|
||||
}
|
||||
ginx.NewRender(c).Data(m, nil)
|
||||
ginx.NewRender(c).Data(event, nil)
|
||||
}
|
||||
|
||||
// 测试事件处理器
|
||||
@@ -181,11 +177,11 @@ func (rt *Router) tryRunEventProcessor(c *gin.Context) {
|
||||
|
||||
processor, err := models.GetProcessorByType(f.ProcessorConfig.Typ, f.ProcessorConfig.Config)
|
||||
if err != nil {
|
||||
ginx.Bomb(200, "get processor err: %+v", err)
|
||||
ginx.Bomb(http.StatusBadRequest, "get processor err: %+v", err)
|
||||
}
|
||||
event, res, err := processor.Process(rt.Ctx, event)
|
||||
if err != nil {
|
||||
ginx.Bomb(200, "processor err: %+v", err)
|
||||
ginx.Bomb(http.StatusBadRequest, "processor err: %+v", err)
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data(map[string]interface{}{
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"math"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -12,7 +13,6 @@ import (
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/toolkits/pkg/ginx"
|
||||
"github.com/toolkits/pkg/i18n"
|
||||
)
|
||||
|
||||
// Return all, front-end search and paging
|
||||
@@ -71,15 +71,14 @@ func (rt *Router) alertMuteAdd(c *gin.Context) {
|
||||
}
|
||||
|
||||
type MuteTestForm struct {
|
||||
EventId int64 `json:"event_id" binding:"required"`
|
||||
AlertMute models.AlertMute `json:"config" binding:"required"`
|
||||
PassTimeCheck bool `json:"pass_time_check"`
|
||||
EventId int64 `json:"event_id" binding:"required"`
|
||||
AlertMute models.AlertMute `json:"mute_config" binding:"required"`
|
||||
}
|
||||
|
||||
func (rt *Router) alertMuteTryRun(c *gin.Context) {
|
||||
|
||||
var f MuteTestForm
|
||||
ginx.BindJSON(c, &f)
|
||||
ginx.Dangerous(f.AlertMute.Verify())
|
||||
|
||||
hisEvent, err := models.AlertHisEventGetById(rt.Ctx, f.EventId)
|
||||
ginx.Dangerous(err)
|
||||
@@ -91,30 +90,18 @@ func (rt *Router) alertMuteTryRun(c *gin.Context) {
|
||||
curEvent := *hisEvent.ToCur()
|
||||
curEvent.SetTagsMap()
|
||||
|
||||
if f.PassTimeCheck {
|
||||
f.AlertMute.MuteTimeType = models.Periodic
|
||||
f.AlertMute.PeriodicMutesJson = []models.PeriodicMute{
|
||||
{
|
||||
EnableDaysOfWeek: "0 1 2 3 4 5 6",
|
||||
EnableStime: "00:00",
|
||||
EnableEtime: "00:00",
|
||||
},
|
||||
}
|
||||
}
|
||||
// 绕过时间范围检查：设置时间范围为全量（0 到 int64 最大值），仅验证其他匹配条件（如标签、策略类型等）
|
||||
f.AlertMute.MuteTimeType = models.TimeRange
|
||||
f.AlertMute.Btime = 0 // 最小可能值（如 Unix 时间戳起点）
|
||||
f.AlertMute.Etime = math.MaxInt64 // 最大可能值（int64 上限）
|
||||
|
||||
match, err := mute.MatchMute(&curEvent, &f.AlertMute)
|
||||
if err != nil {
|
||||
// 对错误信息进行 i18n 翻译
|
||||
translatedErr := i18n.Sprintf(c.GetHeader("X-Language"), err.Error())
|
||||
ginx.Bomb(http.StatusBadRequest, translatedErr)
|
||||
}
|
||||
|
||||
if !match {
|
||||
ginx.NewRender(c).Data("event not match mute", nil)
|
||||
if !mute.MatchMute(&curEvent, &f.AlertMute) {
|
||||
ginx.NewRender(c).Data("not match", nil)
|
||||
return
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data("event match mute", nil)
|
||||
ginx.NewRender(c).Data("mute test match", nil)
|
||||
|
||||
}
|
||||
|
||||
// Preview events (alert_cur_event) that match the mute strategy based on the following criteria:
|
||||
|
||||
@@ -6,12 +6,11 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/alert/dispatch"
|
||||
"github.com/ccfos/nightingale/v6/memsto"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
"github.com/ccfos/nightingale/v6/pkg/ctx"
|
||||
"github.com/ccfos/nightingale/v6/pkg/slice"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/pkg/errors"
|
||||
"github.com/toolkits/pkg/ginx"
|
||||
"github.com/toolkits/pkg/logger"
|
||||
)
|
||||
@@ -162,107 +161,94 @@ func (rt *Router) notifyTest(c *gin.Context) {
|
||||
ginx.Bomb(http.StatusBadRequest, "not events applicable")
|
||||
}
|
||||
|
||||
resp, err := SendNotifyChannelMessage(rt.Ctx, rt.UserCache, rt.UserGroupCache, f.NotifyConfig, events)
|
||||
ginx.NewRender(c).Data(resp, err)
|
||||
}
|
||||
|
||||
func SendNotifyChannelMessage(ctx *ctx.Context, userCache *memsto.UserCacheType, userGroup *memsto.UserGroupCacheType, notifyConfig models.NotifyConfig, events []*models.AlertCurEvent) (string, error) {
|
||||
notifyChannels, err := models.NotifyChannelGets(ctx, notifyConfig.ChannelID, "", "", -1)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get notify channels: %v", err)
|
||||
}
|
||||
|
||||
notifyChannels, err := models.NotifyChannelGets(rt.Ctx, f.NotifyConfig.ChannelID, "", "", -1)
|
||||
ginx.Dangerous(err)
|
||||
if len(notifyChannels) == 0 {
|
||||
return "", fmt.Errorf("notify channel not found")
|
||||
ginx.Bomb(http.StatusBadRequest, "notify channel not found")
|
||||
}
|
||||
|
||||
notifyChannel := notifyChannels[0]
|
||||
if !notifyChannel.Enable {
|
||||
return "", fmt.Errorf("notify channel not enabled, please enable it first")
|
||||
}
|
||||
tplContent := make(map[string]interface{})
|
||||
if notifyChannel.RequestType != "flashduty" {
|
||||
messageTemplates, err := models.MessageTemplateGets(ctx, notifyConfig.TemplateID, "", "")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get message templates: %v", err)
|
||||
}
|
||||
|
||||
if !notifyChannel.Enable {
|
||||
ginx.Bomb(http.StatusBadRequest, "notify channel not enabled, please enable it first")
|
||||
}
|
||||
|
||||
tplContent := make(map[string]interface{})
|
||||
if notifyChannel.RequestType != "flashtudy" {
|
||||
messageTemplates, err := models.MessageTemplateGets(rt.Ctx, f.NotifyConfig.TemplateID, "", "")
|
||||
ginx.Dangerous(err)
|
||||
if len(messageTemplates) == 0 {
|
||||
return "", fmt.Errorf("message template not found")
|
||||
ginx.Bomb(http.StatusBadRequest, "message template not found")
|
||||
}
|
||||
tplContent = messageTemplates[0].RenderEvent(events)
|
||||
}
|
||||
|
||||
var contactKey string
|
||||
if notifyChannel.ParamConfig != nil && notifyChannel.ParamConfig.UserInfo != nil {
|
||||
contactKey = notifyChannel.ParamConfig.UserInfo.ContactKey
|
||||
}
|
||||
|
||||
sendtos, flashDutyChannelIDs, customParams := dispatch.GetNotifyConfigParams(¬ifyConfig, contactKey, userCache, userGroup)
|
||||
sendtos, flashDutyChannelIDs, customParams := dispatch.GetNotifyConfigParams(&f.NotifyConfig, contactKey, rt.UserCache, rt.UserGroupCache)
|
||||
|
||||
var resp string
|
||||
switch notifyChannel.RequestType {
|
||||
case "flashduty":
|
||||
client, err := models.GetHTTPClient(notifyChannel)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get http client: %v", err)
|
||||
}
|
||||
ginx.Dangerous(err)
|
||||
|
||||
for i := range flashDutyChannelIDs {
|
||||
resp, err = notifyChannel.SendFlashDuty(events, flashDutyChannelIDs[i], client)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to send flashduty notify: %v", err)
|
||||
break
|
||||
}
|
||||
}
|
||||
logger.Infof("channel_name: %v, event:%+v, tplContent:%s, customParams:%v, respBody: %v, err: %v", notifyChannel.Name, events[0], tplContent, customParams, resp, err)
|
||||
return resp, nil
|
||||
ginx.NewRender(c).Data(resp, err)
|
||||
case "http":
|
||||
client, err := models.GetHTTPClient(notifyChannel)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to get http client: %v", err)
|
||||
}
|
||||
ginx.Dangerous(err)
|
||||
|
||||
if notifyChannel.RequestConfig == nil {
|
||||
return "", fmt.Errorf("request config is nil")
|
||||
ginx.Bomb(http.StatusBadRequest, "request config not found")
|
||||
}
|
||||
|
||||
if notifyChannel.RequestConfig.HTTPRequestConfig == nil {
|
||||
return "", fmt.Errorf("http request config is nil")
|
||||
ginx.Bomb(http.StatusBadRequest, "http request config not found")
|
||||
}
|
||||
|
||||
if dispatch.NeedBatchContacts(notifyChannel.RequestConfig.HTTPRequestConfig) || len(sendtos) == 0 {
|
||||
resp, err = notifyChannel.SendHTTP(events, tplContent, customParams, sendtos, client)
|
||||
logger.Infof("channel_name: %v, event:%+v, sendtos:%+v, tplContent:%s, customParams:%v, respBody: %v, err: %v", notifyChannel.Name, events[0], sendtos, tplContent, customParams, resp, err)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to send http notify: %v", err)
|
||||
logger.Errorf("failed to send http notify: %v", err)
|
||||
}
|
||||
return resp, nil
|
||||
ginx.NewRender(c).Data(resp, err)
|
||||
} else {
|
||||
for i := range sendtos {
|
||||
resp, err = notifyChannel.SendHTTP(events, tplContent, customParams, []string{sendtos[i]}, client)
|
||||
logger.Infof("channel_name: %v, event:%+v, tplContent:%s, customParams:%v, sendto:%+v, respBody: %v, err: %v", notifyChannel.Name, events[0], tplContent, customParams, sendtos[i], resp, err)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to send http notify: %v", err)
|
||||
logger.Errorf("failed to send http notify: %v", err)
|
||||
ginx.NewRender(c).Message(err)
|
||||
return
|
||||
}
|
||||
}
|
||||
return resp, nil
|
||||
ginx.NewRender(c).Message(err)
|
||||
}
|
||||
|
||||
case "smtp":
|
||||
if len(sendtos) == 0 {
|
||||
return "", fmt.Errorf("no valid email address in the user and team")
|
||||
ginx.Bomb(http.StatusBadRequest, "No valid email address in the user and team")
|
||||
}
|
||||
err := notifyChannel.SendEmailNow(events, tplContent, sendtos)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to send email notify: %v", err)
|
||||
}
|
||||
return resp, nil
|
||||
ginx.NewRender(c).Message(err)
|
||||
case "script":
|
||||
resp, _, err := notifyChannel.SendScript(events, tplContent, customParams, sendtos)
|
||||
logger.Infof("channel_name: %v, event:%+v, tplContent:%s, customParams:%v, respBody: %v, err: %v", notifyChannel.Name, events[0], tplContent, customParams, resp, err)
|
||||
return resp, err
|
||||
ginx.NewRender(c).Data(resp, err)
|
||||
default:
|
||||
logger.Errorf("unsupported request type: %v", notifyChannel.RequestType)
|
||||
return "", fmt.Errorf("unsupported request type")
|
||||
ginx.NewRender(c).Message(errors.New("unsupported request type"))
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"github.com/ccfos/nightingale/v6/datasource/opensearch"
|
||||
"github.com/ccfos/nightingale/v6/dscache"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/toolkits/pkg/ginx"
|
||||
"github.com/toolkits/pkg/logger"
|
||||
)
|
||||
|
||||
func (rt *Router) QueryOSIndices(c *gin.Context) {
|
||||
var f IndexReq
|
||||
ginx.BindJSON(c, &f)
|
||||
|
||||
plug, exists := dscache.DsCache.Get(f.Cate, f.DatasourceId)
|
||||
if !exists {
|
||||
logger.Warningf("cluster:%d not exists", f.DatasourceId)
|
||||
ginx.Bomb(200, "cluster not exists")
|
||||
}
|
||||
|
||||
indices, err := plug.(*opensearch.OpenSearch).QueryIndices()
|
||||
ginx.Dangerous(err)
|
||||
|
||||
ginx.NewRender(c).Data(indices, nil)
|
||||
}
|
||||
|
||||
func (rt *Router) QueryOSFields(c *gin.Context) {
|
||||
var f IndexReq
|
||||
ginx.BindJSON(c, &f)
|
||||
|
||||
plug, exists := dscache.DsCache.Get(f.Cate, f.DatasourceId)
|
||||
if !exists {
|
||||
logger.Warningf("cluster:%d not exists", f.DatasourceId)
|
||||
ginx.Bomb(200, "cluster not exists")
|
||||
}
|
||||
|
||||
fields, err := plug.(*opensearch.OpenSearch).QueryFields([]string{f.Index})
|
||||
ginx.Dangerous(err)
|
||||
|
||||
ginx.NewRender(c).Data(fields, nil)
|
||||
}
|
||||
|
||||
func (rt *Router) QueryOSVariable(c *gin.Context) {
|
||||
var f FieldValueReq
|
||||
ginx.BindJSON(c, &f)
|
||||
|
||||
plug, exists := dscache.DsCache.Get(f.Cate, f.DatasourceId)
|
||||
if !exists {
|
||||
logger.Warningf("cluster:%d not exists", f.DatasourceId)
|
||||
ginx.Bomb(200, "cluster not exists")
|
||||
}
|
||||
|
||||
fields, err := plug.(*opensearch.OpenSearch).QueryFieldValue([]string{f.Index}, f.Query.Field, f.Query.Query)
|
||||
ginx.Dangerous(err)
|
||||
|
||||
ginx.NewRender(c).Data(fields, nil)
|
||||
}
|
||||
@@ -7,20 +7,16 @@ import (
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httputil"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/pkg/poster"
|
||||
pkgprom "github.com/ccfos/nightingale/v6/pkg/prom"
|
||||
"github.com/ccfos/nightingale/v6/prom"
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/prometheus/common/model"
|
||||
"github.com/toolkits/pkg/ginx"
|
||||
"github.com/toolkits/pkg/logger"
|
||||
"github.com/toolkits/pkg/net/httplib"
|
||||
)
|
||||
|
||||
type QueryFormItem struct {
|
||||
@@ -239,94 +235,3 @@ func transportPut(dsid, updatedat int64, tran http.RoundTripper) {
|
||||
updatedAts[dsid] = updatedat
|
||||
transportsLock.Unlock()
|
||||
}
|
||||
|
||||
const (
|
||||
DatasourceTypePrometheus = "Prometheus"
|
||||
DatasourceTypeVictoriaMetrics = "VictoriaMetrics"
|
||||
)
|
||||
|
||||
type deleteDatasourceSeriesForm struct {
|
||||
DatasourceID int64 `json:"datasource_id"`
|
||||
Match []string `json:"match"`
|
||||
Start string `json:"start"`
|
||||
End string `json:"end"`
|
||||
}
|
||||
|
||||
func (rt *Router) deleteDatasourceSeries(c *gin.Context) {
|
||||
var ddsf deleteDatasourceSeriesForm
|
||||
ginx.BindJSON(c, &ddsf)
|
||||
ds := rt.DatasourceCache.GetById(ddsf.DatasourceID)
|
||||
|
||||
if ds == nil {
|
||||
ginx.Bomb(http.StatusBadRequest, "no such datasource")
|
||||
return
|
||||
}
|
||||
|
||||
// Get datasource type, now only support prometheus and victoriametrics
|
||||
datasourceType, ok := ds.SettingsJson["prometheus.tsdb_type"]
|
||||
if !ok {
|
||||
ginx.Bomb(http.StatusBadRequest, "datasource type not found, please check your datasource settings")
|
||||
return
|
||||
}
|
||||
|
||||
target, err := ds.HTTPJson.ParseUrl()
|
||||
if err != nil {
|
||||
ginx.Bomb(http.StatusInternalServerError, "invalid urls: %s", ds.HTTPJson.GetUrls())
|
||||
return
|
||||
}
|
||||
|
||||
timeout := time.Duration(ds.HTTPJson.DialTimeout) * time.Millisecond
|
||||
matchQuerys := make([]string, 0)
|
||||
for _, match := range ddsf.Match {
|
||||
matchQuerys = append(matchQuerys, fmt.Sprintf("match[]=%s", match))
|
||||
}
|
||||
matchQuery := strings.Join(matchQuerys, "&")
|
||||
|
||||
switch datasourceType {
|
||||
case DatasourceTypePrometheus:
|
||||
// Prometheus delete api need POST method
|
||||
// https://prometheus.io/docs/prometheus/latest/querying/api/#delete-series
|
||||
url := fmt.Sprintf("http://%s/api/v1/admin/tsdb/delete_series?%s&start=%s&end=%s", target.Host, matchQuery, ddsf.Start, ddsf.End)
|
||||
go func() {
|
||||
resp, _, err := poster.PostJSON(url, timeout, nil)
|
||||
if err != nil {
|
||||
logger.Errorf("delete series error datasource_id: %d, datasource_name: %s, match: %s, start: %s, end: %s, err: %v",
|
||||
ddsf.DatasourceID, ds.Name, ddsf.Match, ddsf.Start, ddsf.End, err)
|
||||
return
|
||||
}
|
||||
logger.Infof("delete datasource series datasource_id: %d, datasource_name: %s, match: %s, start: %s, end: %s, respBody: %s",
|
||||
ddsf.DatasourceID, ds.Name, ddsf.Match, ddsf.Start, ddsf.End, string(resp))
|
||||
}()
|
||||
case DatasourceTypeVictoriaMetrics:
|
||||
// Delete API doesn’t support the deletion of specific time ranges.
|
||||
// Refer: https://docs.victoriametrics.com/victoriametrics/single-server-victoriametrics/#how-to-delete-time-series
|
||||
var url string
|
||||
// Check VictoriaMetrics is single node or cluster
|
||||
// Cluster will have /select/<accountID>/prometheus pattern
|
||||
re := regexp.MustCompile(`/select/(\d+)/prometheus`)
|
||||
matches := re.FindStringSubmatch(ds.HTTPJson.Url)
|
||||
if len(matches) > 0 && matches[1] != "" {
|
||||
accountID, err := strconv.Atoi(matches[1])
|
||||
if err != nil {
|
||||
ginx.Bomb(http.StatusInternalServerError, "invalid accountID: %s", matches[1])
|
||||
}
|
||||
url = fmt.Sprintf("http://%s/delete/%d/prometheus/api/v1/admin/tsdb/delete_series?%s", target.Host, accountID, matchQuery)
|
||||
} else {
|
||||
url = fmt.Sprintf("http://%s/api/v1/admin/tsdb/delete_series?%s", target.Host, matchQuery)
|
||||
}
|
||||
go func() {
|
||||
resp, err := httplib.Get(url).SetTimeout(timeout).Response()
|
||||
if err != nil {
|
||||
logger.Errorf("delete series failed | datasource_id: %d, datasource_name: %s, match: %s, start: %s, end: %s, err: %v",
|
||||
ddsf.DatasourceID, ds.Name, ddsf.Match, ddsf.Start, ddsf.End, err)
|
||||
return
|
||||
}
|
||||
logger.Infof("sending delete series request | datasource_id: %d, datasource_name: %s, match: %s, start: %s, end: %s, respBody: %s",
|
||||
ddsf.DatasourceID, ds.Name, ddsf.Match, ddsf.Start, ddsf.End, resp.Body)
|
||||
}()
|
||||
default:
|
||||
ginx.Bomb(http.StatusBadRequest, "not support delete series yet")
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data(nil, nil)
|
||||
}
|
||||
|
||||
@@ -235,20 +235,3 @@ func (rt *Router) userDel(c *gin.Context) {
|
||||
|
||||
ginx.NewRender(c).Message(target.Del(rt.Ctx))
|
||||
}
|
||||
|
||||
func (rt *Router) installDateGet(c *gin.Context) {
|
||||
rootUser, err := models.UserGetByUsername(rt.Ctx, "root")
|
||||
if err != nil {
|
||||
logger.Errorf("get root user failed: %v", err)
|
||||
ginx.NewRender(c).Data(0, nil)
|
||||
return
|
||||
}
|
||||
|
||||
if rootUser == nil {
|
||||
logger.Errorf("root user not found")
|
||||
ginx.NewRender(c).Data(0, nil)
|
||||
return
|
||||
}
|
||||
|
||||
ginx.NewRender(c).Data(rootUser.CreateAt, nil)
|
||||
}
|
||||
|
||||
@@ -14,13 +14,6 @@ func decryptConfig(config *ConfigType, cryptoKey string) error {
|
||||
|
||||
config.DB.DSN = decryptDsn
|
||||
|
||||
decryptRedisPwd, err := secu.DealWithDecrypt(config.Redis.Password, cryptoKey)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to decrypt the redis password: %s", err)
|
||||
}
|
||||
|
||||
config.Redis.Password = decryptRedisPwd
|
||||
|
||||
for k := range config.HTTP.APIForService.BasicAuth {
|
||||
decryptPwd, err := secu.DealWithDecrypt(config.HTTP.APIForService.BasicAuth[k], cryptoKey)
|
||||
if err != nil {
|
||||
|
||||
@@ -416,9 +416,7 @@ func QueryData(ctx context.Context, queryParam interface{}, cliTimeout int64, ve
|
||||
MinDocCount(1)
|
||||
|
||||
if strings.HasPrefix(version, "7") {
|
||||
// 添加偏移量，使第一个分桶bucket的左边界对齐为 start 时间
|
||||
offset := (start % param.Interval) - param.Interval
|
||||
tsAggr.FixedInterval(fmt.Sprintf("%ds", param.Interval)).Offset(fmt.Sprintf("%ds", offset))
|
||||
tsAggr.FixedInterval(fmt.Sprintf("%ds", param.Interval))
|
||||
} else {
|
||||
// 兼容 7.0 以下的版本
|
||||
// OpenSearch 也使用这个字段
|
||||
|
||||
@@ -1,199 +0,0 @@
|
||||
package doris
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/datasource"
|
||||
"github.com/ccfos/nightingale/v6/dskit/doris"
|
||||
"github.com/ccfos/nightingale/v6/dskit/types"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
|
||||
"github.com/mitchellh/mapstructure"
|
||||
"github.com/toolkits/pkg/logger"
|
||||
)
|
||||
|
||||
const (
|
||||
DorisType = "doris"
|
||||
)
|
||||
|
||||
func init() {
|
||||
datasource.RegisterDatasource(DorisType, new(Doris))
|
||||
}
|
||||
|
||||
type Doris struct {
|
||||
doris.Doris `json:",inline" mapstructure:",squash"`
|
||||
}
|
||||
|
||||
type QueryParam struct {
|
||||
Ref string `json:"ref" mapstructure:"ref"`
|
||||
Database string `json:"database" mapstructure:"database"`
|
||||
Table string `json:"table" mapstructure:"table"`
|
||||
SQL string `json:"sql" mapstructure:"sql"`
|
||||
Keys datasource.Keys `json:"keys" mapstructure:"keys"`
|
||||
}
|
||||
|
||||
func (d *Doris) InitClient() error {
|
||||
if len(d.Addr) == 0 {
|
||||
return fmt.Errorf("not found doris addr, please check datasource config")
|
||||
}
|
||||
if _, err := d.NewConn(context.TODO(), ""); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Doris) Init(settings map[string]interface{}) (datasource.Datasource, error) {
|
||||
newest := new(Doris)
|
||||
err := mapstructure.Decode(settings, newest)
|
||||
return newest, err
|
||||
}
|
||||
|
||||
func (d *Doris) Validate(ctx context.Context) error {
|
||||
if len(d.Addr) == 0 || len(strings.TrimSpace(d.Addr)) == 0 {
|
||||
return fmt.Errorf("doris addr is invalid, please check datasource setting")
|
||||
}
|
||||
|
||||
if len(strings.TrimSpace(d.User)) == 0 {
|
||||
return fmt.Errorf("doris user is invalid, please check datasource setting")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Equal compares whether two objects are the same, used for caching
|
||||
func (d *Doris) Equal(p datasource.Datasource) bool {
|
||||
newest, ok := p.(*Doris)
|
||||
if !ok {
|
||||
logger.Errorf("unexpected plugin type, expected is ck")
|
||||
return false
|
||||
}
|
||||
|
||||
// only compare first shard
|
||||
if d.Addr != newest.Addr {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.User != newest.User {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.Password != newest.Password {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.EnableWrite != newest.EnableWrite {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.FeAddr != newest.FeAddr {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.MaxQueryRows != newest.MaxQueryRows {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.Timeout != newest.Timeout {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.MaxIdleConns != newest.MaxIdleConns {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.MaxOpenConns != newest.MaxOpenConns {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.ConnMaxLifetime != newest.ConnMaxLifetime {
|
||||
return false
|
||||
}
|
||||
|
||||
if d.ClusterName != newest.ClusterName {
|
||||
return false
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
func (d *Doris) MakeLogQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (d *Doris) MakeTSQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (d *Doris) QueryMapData(ctx context.Context, query interface{}) ([]map[string]string, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func (d *Doris) QueryData(ctx context.Context, query interface{}) ([]models.DataResp, error) {
|
||||
dorisQueryParam := new(QueryParam)
|
||||
if err := mapstructure.Decode(query, dorisQueryParam); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if dorisQueryParam.Keys.ValueKey == "" {
|
||||
return nil, fmt.Errorf("valueKey is required")
|
||||
}
|
||||
|
||||
items, err := d.QueryTimeseries(context.TODO(), &doris.QueryParam{
|
||||
Database: dorisQueryParam.Database,
|
||||
Sql: dorisQueryParam.SQL,
|
||||
Keys: types.Keys{
|
||||
ValueKey: dorisQueryParam.Keys.ValueKey,
|
||||
LabelKey: dorisQueryParam.Keys.LabelKey,
|
||||
TimeKey: dorisQueryParam.Keys.TimeKey,
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
logger.Warningf("query:%+v get data err:%v", dorisQueryParam, err)
|
||||
return []models.DataResp{}, err
|
||||
}
|
||||
data := make([]models.DataResp, 0)
|
||||
for i := range items {
|
||||
data = append(data, models.DataResp{
|
||||
Ref: dorisQueryParam.Ref,
|
||||
Metric: items[i].Metric,
|
||||
Values: items[i].Values,
|
||||
})
|
||||
}
|
||||
|
||||
// parse resp to time series data
|
||||
logger.Infof("req:%+v keys:%+v \n data:%v", dorisQueryParam, dorisQueryParam.Keys, data)
|
||||
|
||||
return data, nil
|
||||
}
|
||||
|
||||
func (d *Doris) QueryLog(ctx context.Context, query interface{}) ([]interface{}, int64, error) {
|
||||
dorisQueryParam := new(QueryParam)
|
||||
if err := mapstructure.Decode(query, dorisQueryParam); err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
|
||||
items, err := d.QueryLogs(ctx, &doris.QueryParam{
|
||||
Database: dorisQueryParam.Database,
|
||||
Sql: dorisQueryParam.SQL,
|
||||
})
|
||||
if err != nil {
|
||||
logger.Warningf("query:%+v get data err:%v", dorisQueryParam, err)
|
||||
return []interface{}{}, 0, err
|
||||
}
|
||||
logs := make([]interface{}, 0)
|
||||
for i := range items {
|
||||
logs = append(logs, items[i])
|
||||
}
|
||||
|
||||
return logs, 0, nil
|
||||
}
|
||||
|
||||
func (d *Doris) DescribeTable(ctx context.Context, query interface{}) ([]*types.ColumnProperty, error) {
|
||||
dorisQueryParam := new(QueryParam)
|
||||
if err := mapstructure.Decode(query, dorisQueryParam); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return d.DescTable(ctx, dorisQueryParam.Database, dorisQueryParam.Table)
|
||||
}
|
||||
@@ -1,399 +0,0 @@
|
||||
package opensearch
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/datasource"
|
||||
"github.com/ccfos/nightingale/v6/datasource/commons/eslike"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
"github.com/ccfos/nightingale/v6/pkg/tlsx"
|
||||
|
||||
"github.com/mitchellh/mapstructure"
|
||||
"github.com/olivere/elastic/v7"
|
||||
oscliv2 "github.com/opensearch-project/opensearch-go/v2"
|
||||
osapiv2 "github.com/opensearch-project/opensearch-go/v2/opensearchapi"
|
||||
)
|
||||
|
||||
const (
|
||||
OpenSearchType = "opensearch"
|
||||
)
|
||||
|
||||
type OpenSearch struct {
|
||||
Addr string `json:"os.addr" mapstructure:"os.addr"`
|
||||
Nodes []string `json:"os.nodes" mapstructure:"os.nodes"`
|
||||
Timeout int64 `json:"os.timeout" mapstructure:"os.timeout"` // millis
|
||||
Basic BasicAuth `json:"os.basic" mapstructure:"os.basic"`
|
||||
TLS TLS `json:"os.tls" mapstructure:"os.tls"`
|
||||
Version string `json:"os.version" mapstructure:"os.version"`
|
||||
Headers map[string]string `json:"os.headers" mapstructure:"os.headers"`
|
||||
MinInterval int `json:"os.min_interval" mapstructure:"os.min_interval"` // seconds
|
||||
MaxShard int `json:"os.max_shard" mapstructure:"os.max_shard"`
|
||||
ClusterName string `json:"os.cluster_name" mapstructure:"os.cluster_name"`
|
||||
Client *oscliv2.Client `json:"os.client" mapstructure:"os.client"`
|
||||
}
|
||||
|
||||
type TLS struct {
|
||||
SkipTlsVerify bool `json:"os.tls.skip_tls_verify" mapstructure:"os.tls.skip_tls_verify"`
|
||||
}
|
||||
|
||||
type BasicAuth struct {
|
||||
Enable bool `json:"os.auth.enable" mapstructure:"os.auth.enable"`
|
||||
Username string `json:"os.user" mapstructure:"os.user"`
|
||||
Password string `json:"os.password" mapstructure:"os.password"`
|
||||
}
|
||||
|
||||
func init() {
|
||||
datasource.RegisterDatasource(OpenSearchType, new(OpenSearch))
|
||||
}
|
||||
|
||||
func (os *OpenSearch) Init(settings map[string]interface{}) (datasource.Datasource, error) {
|
||||
newest := new(OpenSearch)
|
||||
err := mapstructure.Decode(settings, newest)
|
||||
return newest, err
|
||||
}
|
||||
|
||||
func (os *OpenSearch) InitClient() error {
|
||||
transport := &http.Transport{
|
||||
Proxy: http.ProxyFromEnvironment,
|
||||
DialContext: (&net.Dialer{
|
||||
Timeout: time.Duration(os.Timeout) * time.Millisecond,
|
||||
}).DialContext,
|
||||
ResponseHeaderTimeout: time.Duration(os.Timeout) * time.Millisecond,
|
||||
}
|
||||
|
||||
if len(os.Nodes) > 0 {
|
||||
os.Addr = os.Nodes[0]
|
||||
}
|
||||
|
||||
if strings.Contains(os.Addr, "https") {
|
||||
tlsConfig := tlsx.ClientConfig{
|
||||
InsecureSkipVerify: os.TLS.SkipTlsVerify,
|
||||
UseTLS: true,
|
||||
}
|
||||
cfg, err := tlsConfig.TLSConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
transport.TLSClientConfig = cfg
|
||||
}
|
||||
|
||||
headers := http.Header{}
|
||||
for k, v := range os.Headers {
|
||||
headers[k] = []string{v}
|
||||
}
|
||||
|
||||
options := oscliv2.Config{
|
||||
Addresses: os.Nodes,
|
||||
Transport: transport,
|
||||
Header: headers,
|
||||
}
|
||||
|
||||
if os.Basic.Enable && os.Basic.Username != "" {
|
||||
options.Username = os.Basic.Username
|
||||
options.Password = os.Basic.Password
|
||||
}
|
||||
|
||||
var err = error(nil)
|
||||
os.Client, err = oscliv2.NewClient(options)
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
func (os *OpenSearch) Equal(other datasource.Datasource) bool {
|
||||
sort.Strings(os.Nodes)
|
||||
sort.Strings(other.(*OpenSearch).Nodes)
|
||||
|
||||
if strings.Join(os.Nodes, ",") != strings.Join(other.(*OpenSearch).Nodes, ",") {
|
||||
return false
|
||||
}
|
||||
|
||||
if os.Basic.Username != other.(*OpenSearch).Basic.Username {
|
||||
return false
|
||||
}
|
||||
|
||||
if os.Basic.Password != other.(*OpenSearch).Basic.Password {
|
||||
return false
|
||||
}
|
||||
|
||||
if os.TLS.SkipTlsVerify != other.(*OpenSearch).TLS.SkipTlsVerify {
|
||||
return false
|
||||
}
|
||||
|
||||
if os.Timeout != other.(*OpenSearch).Timeout {
|
||||
return false
|
||||
}
|
||||
|
||||
if !reflect.DeepEqual(os.Headers, other.(*OpenSearch).Headers) {
|
||||
return false
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
func (os *OpenSearch) Validate(ctx context.Context) (err error) {
|
||||
if len(os.Nodes) == 0 {
|
||||
return fmt.Errorf("need a valid addr")
|
||||
}
|
||||
|
||||
for _, addr := range os.Nodes {
|
||||
_, err = url.Parse(addr)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse addr error: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
if os.Basic.Enable && (len(os.Basic.Username) == 0 || len(os.Basic.Password) == 0) {
|
||||
return fmt.Errorf("need a valid user, password")
|
||||
}
|
||||
|
||||
if os.MaxShard == 0 {
|
||||
os.MaxShard = 5
|
||||
}
|
||||
|
||||
if os.MinInterval < 10 {
|
||||
os.MinInterval = 10
|
||||
}
|
||||
|
||||
if os.Timeout == 0 {
|
||||
os.Timeout = 6000
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(os.Version, "2") {
|
||||
return fmt.Errorf("version must be 2.0+")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (os *OpenSearch) MakeLogQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
|
||||
return eslike.MakeLogQuery(ctx, query, eventTags, start, end)
|
||||
}
|
||||
|
||||
func (os *OpenSearch) MakeTSQuery(ctx context.Context, query interface{}, eventTags []string, start, end int64) (interface{}, error) {
|
||||
return eslike.MakeTSQuery(ctx, query, eventTags, start, end)
|
||||
}
|
||||
|
||||
func search(ctx context.Context, indices []string, source interface{}, timeout int, cli *oscliv2.Client) (*elastic.SearchResult, error) {
|
||||
var body *bytes.Buffer = nil
|
||||
if source != nil {
|
||||
body = new(bytes.Buffer)
|
||||
err := json.NewEncoder(body).Encode(source)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
req := osapiv2.SearchRequest{
|
||||
Index: indices,
|
||||
Body: body,
|
||||
}
|
||||
|
||||
if timeout > 0 {
|
||||
req.Timeout = time.Second * time.Duration(timeout)
|
||||
}
|
||||
|
||||
resp, err := req.Do(ctx, cli)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return nil, fmt.Errorf("opensearch response not 2xx, resp is %v", resp)
|
||||
}
|
||||
|
||||
bs, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
result := new(elastic.SearchResult)
|
||||
err = json.Unmarshal(bs, &result)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (os *OpenSearch) QueryData(ctx context.Context, queryParam interface{}) ([]models.DataResp, error) {
|
||||
|
||||
search := func(ctx context.Context, indices []string, source interface{}, timeout int, maxShard int) (*elastic.SearchResult, error) {
|
||||
return search(ctx, indices, source, timeout, os.Client)
|
||||
}
|
||||
|
||||
return eslike.QueryData(ctx, queryParam, os.Timeout, os.Version, search)
|
||||
}
|
||||
|
||||
func (os *OpenSearch) QueryIndices() ([]string, error) {
|
||||
|
||||
cir := osapiv2.CatIndicesRequest{
|
||||
Format: "json",
|
||||
}
|
||||
|
||||
rsp, err := cir.Do(context.Background(), os.Client)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rsp.Body.Close()
|
||||
|
||||
bs, err := io.ReadAll(rsp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp := make([]struct {
|
||||
Index string `json:"index"`
|
||||
}, 0)
|
||||
|
||||
err = json.Unmarshal(bs, &resp)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var ret []string
|
||||
for _, k := range resp {
|
||||
ret = append(ret, k.Index)
|
||||
}
|
||||
|
||||
return ret, nil
|
||||
}
|
||||
|
||||
func (os *OpenSearch) QueryFields(indices []string) ([]string, error) {
|
||||
|
||||
var fields []string
|
||||
mappingRequest := osapiv2.IndicesGetMappingRequest{
|
||||
Index: indices,
|
||||
}
|
||||
|
||||
resp, err := mappingRequest.Do(context.Background(), os.Client)
|
||||
if err != nil {
|
||||
return fields, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
bs, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return fields, err
|
||||
}
|
||||
|
||||
result := map[string]interface{}{}
|
||||
|
||||
err = json.Unmarshal(bs, &result)
|
||||
if err != nil {
|
||||
return fields, err
|
||||
}
|
||||
|
||||
idx := ""
|
||||
if len(indices) > 0 {
|
||||
idx = indices[0]
|
||||
}
|
||||
|
||||
mappingIndex := ""
|
||||
indexReg, _ := regexp.Compile(idx)
|
||||
for key, value := range result {
|
||||
mappings, ok := value.(map[string]interface{})
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if len(mappings) == 0 {
|
||||
continue
|
||||
}
|
||||
if key == idx || strings.Contains(key, idx) ||
|
||||
(indexReg != nil && indexReg.MatchString(key)) {
|
||||
mappingIndex = key
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if len(mappingIndex) == 0 {
|
||||
return fields, nil
|
||||
}
|
||||
|
||||
fields = propertyMappingRange(result[mappingIndex], 1)
|
||||
|
||||
sort.Strings(fields)
|
||||
return fields, nil
|
||||
}
|
||||
|
||||
func propertyMappingRange(v interface{}, depth int) (fields []string) {
|
||||
mapping, ok := v.(map[string]interface{})
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if len(mapping) == 0 {
|
||||
return
|
||||
}
|
||||
for key, value := range mapping {
|
||||
if reflect.TypeOf(value).Kind() == reflect.Map {
|
||||
valueMap := value.(map[string]interface{})
|
||||
if prop, found := valueMap["properties"]; found {
|
||||
subFields := propertyMappingRange(prop, depth+1)
|
||||
for i := range subFields {
|
||||
if depth == 1 {
|
||||
fields = append(fields, subFields[i])
|
||||
} else {
|
||||
fields = append(fields, key+"."+subFields[i])
|
||||
}
|
||||
}
|
||||
} else if typ, found := valueMap["type"]; found {
|
||||
if eslike.HitFilter(typ.(string)) {
|
||||
continue
|
||||
}
|
||||
fields = append(fields, key)
|
||||
}
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
func (os *OpenSearch) QueryLog(ctx context.Context, queryParam interface{}) ([]interface{}, int64, error) {
|
||||
|
||||
search := func(ctx context.Context, indices []string, source interface{}, timeout int, maxShard int) (*elastic.SearchResult, error) {
|
||||
return search(ctx, indices, source, timeout, os.Client)
|
||||
}
|
||||
|
||||
return eslike.QueryLog(ctx, queryParam, os.Timeout, os.Version, 0, search)
|
||||
}
|
||||
|
||||
func (os *OpenSearch) QueryFieldValue(indexs []string, field string, query string) ([]string, error) {
|
||||
var values []string
|
||||
source := elastic.NewSearchSource().
|
||||
Size(0)
|
||||
|
||||
if query != "" {
|
||||
source = source.Query(elastic.NewBoolQuery().Must(elastic.NewQueryStringQuery(query)))
|
||||
}
|
||||
source = source.Aggregation("distinct", elastic.NewTermsAggregation().Field(field).Size(10000))
|
||||
|
||||
result, err := search(context.Background(), indexs, source, 0, os.Client)
|
||||
if err != nil {
|
||||
return values, err
|
||||
}
|
||||
|
||||
agg, found := result.Aggregations.Terms("distinct")
|
||||
if !found {
|
||||
return values, nil
|
||||
}
|
||||
|
||||
for _, bucket := range agg.Buckets {
|
||||
values = append(values, bucket.Key.(string))
|
||||
}
|
||||
|
||||
return values, nil
|
||||
}
|
||||
|
||||
func (os *OpenSearch) QueryMapData(ctx context.Context, query interface{}) ([]map[string]string, error) {
|
||||
return nil, nil
|
||||
}
|
||||
@@ -8,10 +8,8 @@ import (
|
||||
|
||||
"github.com/ccfos/nightingale/v6/datasource"
|
||||
_ "github.com/ccfos/nightingale/v6/datasource/ck"
|
||||
_ "github.com/ccfos/nightingale/v6/datasource/doris"
|
||||
"github.com/ccfos/nightingale/v6/datasource/es"
|
||||
_ "github.com/ccfos/nightingale/v6/datasource/mysql"
|
||||
_ "github.com/ccfos/nightingale/v6/datasource/opensearch"
|
||||
_ "github.com/ccfos/nightingale/v6/datasource/postgresql"
|
||||
"github.com/ccfos/nightingale/v6/dskit/tdengine"
|
||||
"github.com/ccfos/nightingale/v6/models"
|
||||
@@ -84,6 +82,8 @@ func getDatasourcesFromDBLoop(ctx *ctx.Context, fromAPI bool) {
|
||||
|
||||
if item.PluginType == "elasticsearch" {
|
||||
esN9eToDatasourceInfo(&ds, item)
|
||||
} else if item.PluginType == "opensearch" {
|
||||
osN9eToDatasourceInfo(&ds, item)
|
||||
} else if item.PluginType == "tdengine" {
|
||||
tdN9eToDatasourceInfo(&ds, item)
|
||||
} else {
|
||||
@@ -144,6 +144,24 @@ func esN9eToDatasourceInfo(ds *datasource.DatasourceInfo, item models.Datasource
|
||||
ds.Settings["es.enable_write"] = item.SettingsJson["enable_write"]
|
||||
}
|
||||
|
||||
// for opensearch
|
||||
func osN9eToDatasourceInfo(ds *datasource.DatasourceInfo, item models.Datasource) {
|
||||
ds.Settings = make(map[string]interface{})
|
||||
ds.Settings["os.nodes"] = []string{item.HTTPJson.Url}
|
||||
ds.Settings["os.timeout"] = item.HTTPJson.Timeout
|
||||
ds.Settings["os.basic"] = es.BasicAuth{
|
||||
Username: item.AuthJson.BasicAuthUser,
|
||||
Password: item.AuthJson.BasicAuthPassword,
|
||||
}
|
||||
ds.Settings["os.tls"] = es.TLS{
|
||||
SkipTlsVerify: item.HTTPJson.TLS.SkipTlsVerify,
|
||||
}
|
||||
ds.Settings["os.version"] = item.SettingsJson["version"]
|
||||
ds.Settings["os.headers"] = item.HTTPJson.Headers
|
||||
ds.Settings["os.min_interval"] = item.SettingsJson["min_interval"]
|
||||
ds.Settings["os.max_shard"] = item.SettingsJson["max_shard"]
|
||||
}
|
||||
|
||||
func PutDatasources(items []datasource.DatasourceInfo) {
|
||||
ids := make([]int64, 0)
|
||||
for _, item := range items {
|
||||
|
||||
@@ -1,543 +0,0 @@
|
||||
package doris
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/dskit/pool"
|
||||
"github.com/ccfos/nightingale/v6/dskit/types"
|
||||
|
||||
_ "github.com/go-sql-driver/mysql" // MySQL driver
|
||||
"github.com/mitchellh/mapstructure"
|
||||
)
|
||||
|
||||
// Doris struct to hold connection details and the connection object
|
||||
type Doris struct {
|
||||
Addr string `json:"doris.addr" mapstructure:"doris.addr"` // be node
|
||||
FeAddr string `json:"doris.fe_addr" mapstructure:"doris.fe_addr"` // fe node
|
||||
User string `json:"doris.user" mapstructure:"doris.user"` //
|
||||
Password string `json:"doris.password" mapstructure:"doris.password"` //
|
||||
Timeout int `json:"doris.timeout" mapstructure:"doris.timeout"`
|
||||
MaxIdleConns int `json:"doris.max_idle_conns" mapstructure:"doris.max_idle_conns"`
|
||||
MaxOpenConns int `json:"doris.max_open_conns" mapstructure:"doris.max_open_conns"`
|
||||
ConnMaxLifetime int `json:"doris.conn_max_lifetime" mapstructure:"doris.conn_max_lifetime"`
|
||||
MaxQueryRows int `json:"doris.max_query_rows" mapstructure:"doris.max_query_rows"`
|
||||
ClusterName string `json:"doris.cluster_name" mapstructure:"doris.cluster_name"`
|
||||
EnableWrite bool `json:"doris.enable_write" mapstructure:"doris.enable_write"`
|
||||
}
|
||||
|
||||
// NewDorisWithSettings initializes a new Doris instance with the given settings
|
||||
func NewDorisWithSettings(ctx context.Context, settings interface{}) (*Doris, error) {
|
||||
newest := new(Doris)
|
||||
settingsMap := map[string]interface{}{}
|
||||
if reflect.TypeOf(settings).Kind() == reflect.String {
|
||||
if err := json.Unmarshal([]byte(settings.(string)), &settingsMap); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
var assert bool
|
||||
settingsMap, assert = settings.(map[string]interface{})
|
||||
if !assert {
|
||||
return nil, errors.New("settings type invalid")
|
||||
}
|
||||
}
|
||||
if err := mapstructure.Decode(settingsMap, newest); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return newest, nil
|
||||
}
|
||||
|
||||
// NewConn establishes a new connection to Doris
|
||||
func (d *Doris) NewConn(ctx context.Context, database string) (*sql.DB, error) {
|
||||
if len(d.Addr) == 0 {
|
||||
return nil, errors.New("empty fe-node addr")
|
||||
}
|
||||
|
||||
// Set default values similar to postgres implementation
|
||||
if d.Timeout == 0 {
|
||||
d.Timeout = 60
|
||||
}
|
||||
if d.MaxIdleConns == 0 {
|
||||
d.MaxIdleConns = 10
|
||||
}
|
||||
if d.MaxOpenConns == 0 {
|
||||
d.MaxOpenConns = 100
|
||||
}
|
||||
if d.ConnMaxLifetime == 0 {
|
||||
d.ConnMaxLifetime = 14400
|
||||
}
|
||||
if d.MaxQueryRows == 0 {
|
||||
d.MaxQueryRows = 500
|
||||
}
|
||||
|
||||
var keys []string
|
||||
keys = append(keys, d.Addr)
|
||||
keys = append(keys, d.Password, d.User)
|
||||
if len(database) > 0 {
|
||||
keys = append(keys, database)
|
||||
}
|
||||
cachedkey := strings.Join(keys, ":")
|
||||
// cache conn with database
|
||||
conn, ok := pool.PoolClient.Load(cachedkey)
|
||||
if ok {
|
||||
return conn.(*sql.DB), nil
|
||||
}
|
||||
var db *sql.DB
|
||||
var err error
|
||||
defer func() {
|
||||
if db != nil && err == nil {
|
||||
pool.PoolClient.Store(cachedkey, db)
|
||||
}
|
||||
}()
|
||||
|
||||
// Simplified connection logic for Doris using MySQL driver
|
||||
dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?charset=utf8", d.User, d.Password, d.Addr, database)
|
||||
db, err = sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set connection pool configuration
|
||||
db.SetMaxIdleConns(d.MaxIdleConns)
|
||||
db.SetMaxOpenConns(d.MaxOpenConns)
|
||||
db.SetConnMaxLifetime(time.Duration(d.ConnMaxLifetime) * time.Second)
|
||||
|
||||
return db, nil
|
||||
}
|
||||
|
||||
// createTimeoutContext creates a context with timeout based on Doris configuration
|
||||
func (d *Doris) createTimeoutContext(ctx context.Context) (context.Context, context.CancelFunc) {
|
||||
timeout := d.Timeout
|
||||
if timeout == 0 {
|
||||
timeout = 60
|
||||
}
|
||||
return context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
|
||||
}
|
||||
|
||||
// ShowDatabases lists all databases in Doris
|
||||
func (d *Doris) ShowDatabases(ctx context.Context) ([]string, error) {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
db, err := d.NewConn(timeoutCtx, "")
|
||||
if err != nil {
|
||||
return []string{}, err
|
||||
}
|
||||
|
||||
rows, err := db.QueryContext(timeoutCtx, "SHOW DATABASES")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var databases []string
|
||||
for rows.Next() {
|
||||
var dbName string
|
||||
if err := rows.Scan(&dbName); err != nil {
|
||||
continue
|
||||
}
|
||||
databases = append(databases, dbName)
|
||||
}
|
||||
return databases, nil
|
||||
}
|
||||
|
||||
// ShowResources lists all resources with type resourceType in Doris
|
||||
func (d *Doris) ShowResources(ctx context.Context, resourceType string) ([]string, error) {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
db, err := d.NewConn(timeoutCtx, "")
|
||||
if err != nil {
|
||||
return []string{}, err
|
||||
}
|
||||
|
||||
// 使用 SHOW RESOURCES 命令
|
||||
query := fmt.Sprintf("SHOW RESOURCES WHERE RESOURCETYPE = '%s'", resourceType)
|
||||
rows, err := db.QueryContext(timeoutCtx, query)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to execute query: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
distinctName := make(map[string]struct{})
|
||||
|
||||
// 获取列信息
|
||||
columns, err := rows.Columns()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to get columns: %w", err)
|
||||
}
|
||||
|
||||
// 准备接收数据的变量
|
||||
values := make([]interface{}, len(columns))
|
||||
valuePtrs := make([]interface{}, len(columns))
|
||||
for i := range values {
|
||||
valuePtrs[i] = &values[i]
|
||||
}
|
||||
|
||||
// 遍历结果集
|
||||
for rows.Next() {
|
||||
err := rows.Scan(valuePtrs...)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("error scanning row: %w", err)
|
||||
}
|
||||
// 提取资源名称并添加到 map 中（自动去重）
|
||||
if name, ok := values[0].([]byte); ok {
|
||||
distinctName[string(name)] = struct{}{}
|
||||
} else if nameStr, ok := values[0].(string); ok {
|
||||
distinctName[nameStr] = struct{}{}
|
||||
}
|
||||
}
|
||||
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, fmt.Errorf("error iterating rows: %w", err)
|
||||
}
|
||||
|
||||
// 将 map 转换为切片
|
||||
var resources []string
|
||||
for name := range distinctName {
|
||||
resources = append(resources, name)
|
||||
}
|
||||
|
||||
return resources, nil
|
||||
}
|
||||
|
||||
// ShowTables lists all tables in a given database
|
||||
func (d *Doris) ShowTables(ctx context.Context, database string) ([]string, error) {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
db, err := d.NewConn(timeoutCtx, database)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
query := fmt.Sprintf("SHOW TABLES IN %s", database)
|
||||
rows, err := db.QueryContext(timeoutCtx, query)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var tables []string
|
||||
for rows.Next() {
|
||||
var tableName string
|
||||
if err := rows.Scan(&tableName); err != nil {
|
||||
continue
|
||||
}
|
||||
tables = append(tables, tableName)
|
||||
}
|
||||
return tables, nil
|
||||
}
|
||||
|
||||
// DescTable describes the schema of a specified table in Doris
|
||||
func (d *Doris) DescTable(ctx context.Context, database, table string) ([]*types.ColumnProperty, error) {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
db, err := d.NewConn(timeoutCtx, database)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
query := fmt.Sprintf("DESCRIBE %s.%s", database, table)
|
||||
rows, err := db.QueryContext(timeoutCtx, query)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
// 日志报表中需要把 .type 转化成内部类型
|
||||
// TODO: 是否有复合类型, Array/JSON/Tuple/Nested, 是否有更多的类型
|
||||
convertDorisType := func(origin string) (string, bool) {
|
||||
lower := strings.ToLower(origin)
|
||||
switch lower {
|
||||
case "double":
|
||||
return types.LogExtractValueTypeFloat, true
|
||||
|
||||
case "datetime", "date":
|
||||
return types.LogExtractValueTypeDate, false
|
||||
|
||||
case "text":
|
||||
return types.LogExtractValueTypeText, true
|
||||
|
||||
default:
|
||||
if strings.Contains(lower, "int") {
|
||||
return types.LogExtractValueTypeLong, true
|
||||
}
|
||||
// 日期类型统一按照.date处理
|
||||
if strings.HasPrefix(lower, "date") {
|
||||
return types.LogExtractValueTypeDate, false
|
||||
}
|
||||
if strings.HasPrefix(lower, "varchar") || strings.HasPrefix(lower, "char") {
|
||||
return types.LogExtractValueTypeText, true
|
||||
}
|
||||
if strings.HasPrefix(lower, "decimal") {
|
||||
return types.LogExtractValueTypeFloat, true
|
||||
}
|
||||
}
|
||||
|
||||
return origin, false
|
||||
}
|
||||
|
||||
var columns []*types.ColumnProperty
|
||||
for rows.Next() {
|
||||
var (
|
||||
field string
|
||||
typ string
|
||||
null string
|
||||
key string
|
||||
defaultValue sql.NullString
|
||||
extra string
|
||||
)
|
||||
if err := rows.Scan(&field, &typ, &null, &key, &defaultValue, &extra); err != nil {
|
||||
continue
|
||||
}
|
||||
type2, indexable := convertDorisType(typ)
|
||||
columns = append(columns, &types.ColumnProperty{
|
||||
Field: field,
|
||||
Type: typ, // You might want to convert MySQL types to your custom types
|
||||
|
||||
Type2: type2,
|
||||
Indexable: indexable,
|
||||
})
|
||||
}
|
||||
return columns, nil
|
||||
}
|
||||
|
||||
// SelectRows selects rows from a specified table in Doris based on a given query with MaxQueryRows check
|
||||
func (d *Doris) SelectRows(ctx context.Context, database, table, query string) ([]map[string]interface{}, error) {
|
||||
sql := fmt.Sprintf("SELECT * FROM %s.%s", database, table)
|
||||
if query != "" {
|
||||
sql += " " + query
|
||||
}
|
||||
|
||||
// 检查查询结果行数
|
||||
err := d.CheckMaxQueryRows(ctx, database, sql)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return d.ExecQuery(ctx, database, sql)
|
||||
}
|
||||
|
||||
// ExecQuery executes a given SQL query in Doris and returns the results
|
||||
func (d *Doris) ExecQuery(ctx context.Context, database string, sql string) ([]map[string]interface{}, error) {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
db, err := d.NewConn(timeoutCtx, database)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err := db.QueryContext(timeoutCtx, sql)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
columns, err := rows.Columns()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var results []map[string]interface{}
|
||||
|
||||
for rows.Next() {
|
||||
columnValues := make([]interface{}, len(columns))
|
||||
columnPointers := make([]interface{}, len(columns))
|
||||
for i := range columnValues {
|
||||
columnPointers[i] = &columnValues[i]
|
||||
}
|
||||
|
||||
if err := rows.Scan(columnPointers...); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
rowMap := make(map[string]interface{})
|
||||
for i, colName := range columns {
|
||||
val := columnValues[i]
|
||||
bytes, ok := val.([]byte)
|
||||
if ok {
|
||||
rowMap[colName] = string(bytes)
|
||||
} else {
|
||||
rowMap[colName] = val
|
||||
}
|
||||
}
|
||||
results = append(results, rowMap)
|
||||
}
|
||||
return results, nil
|
||||
}
|
||||
|
||||
// ExecContext executes a given SQL query in Doris and returns the results
|
||||
func (d *Doris) ExecContext(ctx context.Context, database string, sql string) error {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
db, err := d.NewConn(timeoutCtx, database)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = db.ExecContext(timeoutCtx, sql)
|
||||
return err
|
||||
}
|
||||
|
||||
// ExecBatchSQL 执行多条 SQL 语句
|
||||
func (d *Doris) ExecBatchSQL(ctx context.Context, database string, sqlBatch string) error {
|
||||
// 分割 SQL 语句
|
||||
sqlStatements := SplitSQLStatements(sqlBatch)
|
||||
|
||||
// 逐条执行 SQL 语句
|
||||
for _, ql := range sqlStatements {
|
||||
// 跳过空语句
|
||||
ql = strings.TrimSpace(ql)
|
||||
if ql == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// 检查是否是 CREATE DATABASE 语句
|
||||
isCreateDB := strings.HasPrefix(strings.ToUpper(ql), "CREATE DATABASE")
|
||||
// strings.HasPrefix(strings.ToUpper(sql), "CREATE SCHEMA") // 暂时不支持CREATE SCHEMA
|
||||
|
||||
// 对于 CREATE DATABASE 语句，使用空数据库名连接
|
||||
currentDB := database
|
||||
if isCreateDB {
|
||||
currentDB = ""
|
||||
}
|
||||
|
||||
// 执行单条 SQL，ExecContext 内部已经包含超时处理
|
||||
err := d.ExecContext(ctx, currentDB, ql)
|
||||
if err != nil {
|
||||
return fmt.Errorf("exec sql failed, sql:%s, err:%w", sqlBatch, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// SplitSQLStatements 将多条 SQL 语句分割成单独的语句
|
||||
func SplitSQLStatements(sqlBatch string) []string {
|
||||
var statements []string
|
||||
var currentStatement strings.Builder
|
||||
|
||||
// 状态标记
|
||||
var (
|
||||
inString bool // 是否在字符串内
|
||||
inComment bool // 是否在单行注释内
|
||||
inMultilineComment bool // 是否在多行注释内
|
||||
escaped bool // 前一个字符是否为转义字符
|
||||
)
|
||||
|
||||
for i := 0; i < len(sqlBatch); i++ {
|
||||
char := sqlBatch[i]
|
||||
currentStatement.WriteByte(char)
|
||||
|
||||
// 处理转义字符
|
||||
if inString && char == '\\' {
|
||||
escaped = !escaped
|
||||
continue
|
||||
}
|
||||
|
||||
// 处理字符串
|
||||
if char == '\'' && !inComment && !inMultilineComment {
|
||||
if !escaped {
|
||||
inString = !inString
|
||||
}
|
||||
escaped = false
|
||||
continue
|
||||
}
|
||||
|
||||
// 处理单行注释
|
||||
if !inString && !inMultilineComment && !inComment && char == '-' && i+1 < len(sqlBatch) && sqlBatch[i+1] == '-' {
|
||||
inComment = true
|
||||
currentStatement.WriteByte(sqlBatch[i+1]) // 写入第二个'-'
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
// 处理多行注释开始
|
||||
if !inString && !inComment && char == '/' && i+1 < len(sqlBatch) && sqlBatch[i+1] == '*' {
|
||||
inMultilineComment = true
|
||||
currentStatement.WriteByte(sqlBatch[i+1]) // 写入'*'
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
// 处理多行注释结束
|
||||
if inMultilineComment && char == '*' && i+1 < len(sqlBatch) && sqlBatch[i+1] == '/' {
|
||||
inMultilineComment = false
|
||||
currentStatement.WriteByte(sqlBatch[i+1]) // 写入'/'
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
// 处理换行符，结束单行注释
|
||||
if inComment && (char == '\n' || char == '\r') {
|
||||
inComment = false
|
||||
}
|
||||
|
||||
// 分割SQL语句
|
||||
if char == ';' && !inString && !inMultilineComment && !inComment {
|
||||
// 收集到分号后面的单行注释（如果有）
|
||||
for j := i + 1; j < len(sqlBatch); j++ {
|
||||
nextChar := sqlBatch[j]
|
||||
|
||||
// 检查是否是注释开始
|
||||
if nextChar == '-' && j+1 < len(sqlBatch) && sqlBatch[j+1] == '-' {
|
||||
// 找到了注释，添加到当前语句
|
||||
currentStatement.WriteByte(nextChar) // 添加'-'
|
||||
currentStatement.WriteByte(sqlBatch[j+1]) // 添加第二个'-'
|
||||
j++
|
||||
|
||||
// 读取直到行尾
|
||||
for k := j + 1; k < len(sqlBatch); k++ {
|
||||
commentChar := sqlBatch[k]
|
||||
currentStatement.WriteByte(commentChar)
|
||||
j = k
|
||||
|
||||
if commentChar == '\n' || commentChar == '\r' {
|
||||
break
|
||||
}
|
||||
}
|
||||
i = j
|
||||
break
|
||||
} else if !isWhitespace(nextChar) {
|
||||
// 非注释且非空白字符，停止收集
|
||||
break
|
||||
} else {
|
||||
// 是空白字符，添加到当前语句
|
||||
currentStatement.WriteByte(nextChar)
|
||||
i = j
|
||||
}
|
||||
}
|
||||
|
||||
statements = append(statements, strings.TrimSpace(currentStatement.String()))
|
||||
currentStatement.Reset()
|
||||
continue
|
||||
}
|
||||
|
||||
escaped = false
|
||||
}
|
||||
|
||||
// 处理最后一条可能没有分号的语句
|
||||
lastStatement := strings.TrimSpace(currentStatement.String())
|
||||
if lastStatement != "" {
|
||||
statements = append(statements, lastStatement)
|
||||
}
|
||||
|
||||
return statements
|
||||
}
|
||||
|
||||
// 判断字符是否为空白字符
|
||||
func isWhitespace(c byte) bool {
|
||||
return unicode.IsSpace(rune(c))
|
||||
}
|
||||
@@ -1,36 +0,0 @@
|
||||
package doris
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// 日志相关的操作
|
||||
const (
|
||||
TimeseriesAggregationTimestamp = "__ts__"
|
||||
)
|
||||
|
||||
// TODO: 待测试, MAP/ARRAY/STRUCT/JSON 等类型能否处理
|
||||
func (d *Doris) QueryLogs(ctx context.Context, query *QueryParam) ([]map[string]interface{}, error) {
|
||||
// 等同于 Query()
|
||||
return d.Query(ctx, query)
|
||||
}
|
||||
|
||||
// 本质是查询时序数据, 取第一组, SQL由上层封装, 不再做复杂的解析和截断
|
||||
func (d *Doris) QueryHistogram(ctx context.Context, query *QueryParam) ([][]float64, error) {
|
||||
values, err := d.QueryTimeseries(ctx, query)
|
||||
if err != nil {
|
||||
return [][]float64{}, nil
|
||||
}
|
||||
if len(values) > 0 && len(values[0].Values) > 0 {
|
||||
items := values[0].Values
|
||||
sort.Slice(items, func(i, j int) bool {
|
||||
if len(items[i]) > 0 && len(items[j]) > 0 {
|
||||
return items[i][0] < items[j][0]
|
||||
}
|
||||
return false
|
||||
})
|
||||
return items, nil
|
||||
}
|
||||
return [][]float64{}, nil
|
||||
}
|
||||
@@ -1,126 +0,0 @@
|
||||
## SQL变量
|
||||
|
||||
| 字段名 | 含义 | 使用场景 |
|
||||
| ---- | ---- | ---- |
|
||||
|database|数据库|无|
|
||||
|table|表名||
|
||||
|time_field|时间戳的字段||
|
||||
|query|查询条件|日志原文|
|
||||
|from|开始时间||
|
||||
|to|结束时间||
|
||||
|aggregation|聚合算法|时序图|
|
||||
|field|聚合的字段|时序图|
|
||||
|limit|分页参数|日志原文|
|
||||
|offset|分页参数|日志原文|
|
||||
|interval|直方图的时间粒度|直方图|
|
||||
|
||||
## 日志原文
|
||||
### 直方图
|
||||
|
||||
```
|
||||
# 如何计算interval的值
|
||||
max := 60 // 最多60个柱子
|
||||
interval := ($to-$from) / max
|
||||
interval = interval - interval%10
|
||||
if interval <= 0 {
|
||||
interval = 60
|
||||
}
|
||||
```
|
||||
|
||||
```
|
||||
SELECT count() as cnt,
|
||||
FLOOR(UNIX_TIMESTAMP($time_field) / $interval) * $interval AS __ts__
|
||||
FROM $table
|
||||
WHERE $time_field BETWEEN FROM_UNIXTIME($from) AND FROM_UNIXTIME($to)
|
||||
GROUP BY __ts__;
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"database":"$database",
|
||||
"sql":"$sql",
|
||||
"keys:": {
|
||||
"valueKey":"cnt",
|
||||
"timeKey":"__ts__"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 日志原文
|
||||
|
||||
```
|
||||
SELECT * from $table
|
||||
WHERE $time_field BETWEEN FROM_UNIXTIME($from) AND FROM_UNIXTIME($to)
|
||||
ORDER by $time_filed
|
||||
LIMIT $limit OFFSET $offset;
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"database":"$database",
|
||||
"sql":"$sql"
|
||||
}
|
||||
```
|
||||
|
||||
## 时序图
|
||||
|
||||
### 日志行数
|
||||
|
||||
```
|
||||
SELECT COUNT() AS cnt, DATE_FORMAT(date, '%Y-%m-%d %H:%i:00') AS __ts__
|
||||
FROM nginx_access_log
|
||||
WHERE $time_field BETWEEN FROM_UNIXTIME($from) AND FROM_UNIXTIME($to)
|
||||
GROUP BY __ts__
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"database":"$database",
|
||||
"sql":"$sql",
|
||||
"keys:": {
|
||||
"valueKey":"cnt",
|
||||
"timeKey":"__ts__"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### max/min/avg/sum
|
||||
|
||||
```
|
||||
SELECT $aggregation($field) AS series, DATE_FORMAT(date, '%Y-%m-%d %H:%i:00') AS __ts__
|
||||
FROM nginx_access_log
|
||||
WHERE $time_field BETWEEN FROM_UNIXTIME($from) AND FROM_UNIXTIME($to)
|
||||
GROUP BY __ts__
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"database":"$database",
|
||||
"sql":"$sql",
|
||||
"keys:": {
|
||||
"valueKey":"series",
|
||||
"timeKey":"__ts__"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
### 分位值
|
||||
|
||||
```
|
||||
SELECT percentile($field, 0.95) AS series, DATE_FORMAT(date, '%Y-%m-%d %H:%i:00') AS __ts__
|
||||
FROM nginx_access_log
|
||||
WHERE $time_field BETWEEN FROM_UNIXTIME($from) AND FROM_UNIXTIME($to)
|
||||
GROUP BY __ts__
|
||||
```
|
||||
|
||||
```
|
||||
{
|
||||
"database":"$database",
|
||||
"sql":"$sql",
|
||||
"keys:": {
|
||||
"valueKey":"series",
|
||||
"timeKey":"__ts__"
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -1,108 +0,0 @@
|
||||
package doris
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/ccfos/nightingale/v6/dskit/sqlbase"
|
||||
"github.com/ccfos/nightingale/v6/dskit/types"
|
||||
)
|
||||
|
||||
const (
|
||||
TimeFieldFormatEpochMilli = "epoch_millis"
|
||||
TimeFieldFormatEpochSecond = "epoch_second"
|
||||
TimeFieldFormatDateTime = "datetime"
|
||||
)
|
||||
|
||||
// 不再拼接SQL, 完全信赖用户的输入
|
||||
type QueryParam struct {
|
||||
Database string `json:"database"`
|
||||
Sql string `json:"sql"`
|
||||
Keys types.Keys `json:"keys" mapstructure:"keys"`
|
||||
}
|
||||
|
||||
var (
|
||||
DorisBannedOp = map[string]struct{}{
|
||||
"CREATE": {},
|
||||
"INSERT": {},
|
||||
"ALTER": {},
|
||||
"REVOKE": {},
|
||||
"DROP": {},
|
||||
"RENAME": {},
|
||||
"ATTACH": {},
|
||||
"DETACH": {},
|
||||
"OPTIMIZE": {},
|
||||
"TRUNCATE": {},
|
||||
"SET": {},
|
||||
}
|
||||
)
|
||||
|
||||
// Query executes a given SQL query in Doris and returns the results with MaxQueryRows check
|
||||
func (d *Doris) Query(ctx context.Context, query *QueryParam) ([]map[string]interface{}, error) {
|
||||
// 校验SQL的合法性, 过滤掉 write请求
|
||||
sqlItem := strings.Split(strings.ToUpper(query.Sql), " ")
|
||||
for _, item := range sqlItem {
|
||||
if _, ok := DorisBannedOp[item]; ok {
|
||||
return nil, fmt.Errorf("operation %s is forbid, only read db, please check your sql", item)
|
||||
}
|
||||
}
|
||||
|
||||
// 检查查询结果行数
|
||||
err := d.CheckMaxQueryRows(ctx, query.Database, query.Sql)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows, err := d.ExecQuery(ctx, query.Database, query.Sql)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
// QueryTimeseries executes a time series data query using the given parameters with MaxQueryRows check
|
||||
func (d *Doris) QueryTimeseries(ctx context.Context, query *QueryParam) ([]types.MetricValues, error) {
|
||||
// 使用 Query 方法执行查询，Query方法内部已包含MaxQueryRows检查
|
||||
rows, err := d.Query(ctx, query)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return sqlbase.FormatMetricValues(query.Keys, rows), nil
|
||||
}
|
||||
|
||||
// CheckMaxQueryRows checks if the query result exceeds the maximum allowed rows
|
||||
func (d *Doris) CheckMaxQueryRows(ctx context.Context, database, sql string) error {
|
||||
timeoutCtx, cancel := d.createTimeoutContext(ctx)
|
||||
defer cancel()
|
||||
|
||||
cleanedSQL := strings.ReplaceAll(sql, ";", "")
|
||||
checkQuery := fmt.Sprintf("SELECT COUNT(*) as count FROM (%s) AS subquery;", cleanedSQL)
|
||||
|
||||
// 执行计数查询
|
||||
results, err := d.ExecQuery(timeoutCtx, database, checkQuery)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(results) > 0 {
|
||||
if count, exists := results[0]["count"]; exists {
|
||||
v, err := sqlbase.ParseFloat64Value(count)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
maxQueryRows := d.MaxQueryRows
|
||||
if maxQueryRows == 0 {
|
||||
maxQueryRows = 500
|
||||
}
|
||||
|
||||
if v > float64(maxQueryRows) {
|
||||
return fmt.Errorf("query result rows count %d exceeds the maximum limit %d", int(v), maxQueryRows)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"crypto/md5"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strconv"
|
||||
@@ -122,11 +121,6 @@ func FormatMetricValues(keys types.Keys, rows []map[string]interface{}, ignoreDe
|
||||
|
||||
// Compile and store the metric values
|
||||
for metricName, value := range metricValue {
|
||||
// NaN 无法执行json.Marshal(), 接口会报错
|
||||
if math.IsNaN(value) {
|
||||
continue
|
||||
}
|
||||
|
||||
metrics := make(model.Metric)
|
||||
var labelsStr []string
|
||||
|
||||
|
||||
3
go.mod
3
go.mod
@@ -27,13 +27,12 @@ require (
|
||||
github.com/jinzhu/copier v0.4.0
|
||||
github.com/json-iterator/go v1.1.12
|
||||
github.com/koding/multiconfig v0.0.0-20171124222453-69c27309b2d7
|
||||
github.com/lib/pq v1.10.9
|
||||
github.com/lib/pq v1.0.0
|
||||
github.com/mailru/easyjson v0.7.7
|
||||
github.com/mattn/go-isatty v0.0.19
|
||||
github.com/mitchellh/mapstructure v1.5.0
|
||||
github.com/mojocn/base64Captcha v1.3.6
|
||||
github.com/olivere/elastic/v7 v7.0.32
|
||||
github.com/opensearch-project/opensearch-go/v2 v2.3.0
|
||||
github.com/patrickmn/go-cache v2.1.0+incompatible
|
||||
github.com/pelletier/go-toml/v2 v2.0.8
|
||||
github.com/pkg/errors v0.9.1
|
||||
|
||||
26
go.sum
26
go.sum
@@ -31,21 +31,8 @@ github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1
|
||||
github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
|
||||
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de h1:FxWPpzIjnTlhPwqqXc4/vE0f7GvRjuAsbW+HOIe8KnA=
|
||||
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw=
|
||||
github.com/aws/aws-sdk-go v1.44.263/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI=
|
||||
github.com/aws/aws-sdk-go v1.44.302 h1:ST3ko6GrJKn3Xi+nAvxjG3uk/V1pW8KC52WLeIxqqNk=
|
||||
github.com/aws/aws-sdk-go v1.44.302/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI=
|
||||
github.com/aws/aws-sdk-go-v2 v1.18.0/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw=
|
||||
github.com/aws/aws-sdk-go-v2/config v1.18.25/go.mod h1:dZnYpD5wTW/dQF0rRNLVypB396zWCcPiBIvdvSWHEg4=
|
||||
github.com/aws/aws-sdk-go-v2/credentials v1.13.24/go.mod h1:jYPYi99wUOPIFi0rhiOvXeSEReVOzBqFNOX5bXYoG2o=
|
||||
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.3/go.mod h1:4Q0UFP0YJf0NrsEuEYHpM9fTSEVnD16Z3uyEF7J9JGM=
|
||||
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.33/go.mod h1:7i0PF1ME/2eUPFcjkVIwq+DOygHEoK92t5cDqNgYbIw=
|
||||
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.27/go.mod h1:UrHnn3QV/d0pBZ6QBAEQcqFLf8FAzLmoUfPVIueOvoM=
|
||||
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.34/go.mod h1:Etz2dj6UHYuw+Xw830KfzCfWGMzqvUTCjUj5b76GVDc=
|
||||
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.27/go.mod h1:EOwBD4J4S5qYszS5/3DpkejfuK+Z5/1uzICfPaZLtqw=
|
||||
github.com/aws/aws-sdk-go-v2/service/sso v1.12.10/go.mod h1:ouy2P4z6sJN70fR3ka3wD3Ro3KezSxU6eKGQI2+2fjI=
|
||||
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.10/go.mod h1:AFvkxc8xfBe8XA+5St5XIHHrQQtkxqrRincx4hmMHOk=
|
||||
github.com/aws/aws-sdk-go-v2/service/sts v1.19.0/go.mod h1:BgQOMsg8av8jset59jelyPW7NoZcZXLVpDsXunGDrk8=
|
||||
github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
|
||||
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
||||
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
|
||||
github.com/bitly/go-simplejson v0.5.1 h1:xgwPbetQScXt1gh9BmoJ6j9JMr3TElvuIyjR8pgdoow=
|
||||
@@ -152,7 +139,6 @@ github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
|
||||
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
|
||||
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||
@@ -203,7 +189,6 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
|
||||
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
|
||||
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
|
||||
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
|
||||
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
|
||||
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
|
||||
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
|
||||
github.com/jpillora/backoff v1.0.0 h1:uvFg412JmmHBHw7iwprIxkPMI+sGQ4kzOWsMeHnm2EA=
|
||||
@@ -235,8 +220,8 @@ github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+
|
||||
github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
|
||||
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
|
||||
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
|
||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/lib/pq v1.0.0 h1:X5PMW56eZitiTeO7tKzZxFCSpbFZJtkMMooicw2us9A=
|
||||
github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
|
||||
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
|
||||
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
|
||||
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
|
||||
@@ -263,8 +248,6 @@ github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
|
||||
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
|
||||
github.com/olivere/elastic/v7 v7.0.32 h1:R7CXvbu8Eq+WlsLgxmKVKPox0oOwAE/2T9Si5BnvK6E=
|
||||
github.com/olivere/elastic/v7 v7.0.32/go.mod h1:c7PVmLe3Fxq77PIfY/bZmxY/TAamBhCzZ8xDOE09a9k=
|
||||
github.com/opensearch-project/opensearch-go/v2 v2.3.0 h1:nQIEMr+A92CkhHrZgUhcfsrZjibvB3APXf2a1VwCmMQ=
|
||||
github.com/opensearch-project/opensearch-go/v2 v2.3.0/go.mod h1:8LDr9FCgUTVoT+5ESjc2+iaZuldqE+23Iq0r1XeNue8=
|
||||
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
|
||||
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
|
||||
github.com/paulmach/orb v0.11.1 h1:3koVegMC4X/WeiXYz9iswopaTwMem53NzTJuTF20JzU=
|
||||
@@ -409,7 +392,6 @@ golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwY
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
|
||||
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
||||
golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco=
|
||||
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
|
||||
golang.org/x/net v0.8.0/go.mod h1:QVkue5JL9kW//ek3r6jTKnTFis1tRmNAW2P1shuFdJc=
|
||||
@@ -437,7 +419,6 @@ golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBc
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
@@ -447,7 +428,6 @@ golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
|
||||
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
|
||||
golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
|
||||
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
|
||||
@@ -456,7 +436,6 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
|
||||
golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
|
||||
golang.org/x/text v0.8.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
|
||||
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
|
||||
@@ -491,7 +470,6 @@ gopkg.in/gomail.v2 v2.0.0-20160411212932-81ebce5c23df/go.mod h1:LRQQ+SO6ZHR7tOkp
|
||||
gopkg.in/square/go-jose.v2 v2.6.0 h1:NGk74WTnPKBNUhNzQX7PYcTLUjoq7mzKk2OKbvwk2iI=
|
||||
gopkg.in/square/go-jose.v2 v2.6.0/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
|
||||
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
|
||||
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
@@ -142,6 +142,7 @@ func (amc *AlertMuteCacheType) syncAlertMutes() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
amc.stats.GaugeCronDuration.WithLabelValues("sync_alert_mutes").Set(float64(ms))
|
||||
amc.stats.GaugeSyncNumber.WithLabelValues("sync_alert_mutes").Set(float64(len(lst)))
|
||||
logger.Infof("timer: sync mutes done, cost: %dms, number: %d", ms, len(lst))
|
||||
dumper.PutSyncRecord("alert_mutes", start.Unix(), ms, len(lst), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -132,6 +132,7 @@ func (arc *AlertRuleCacheType) syncAlertRules() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
arc.stats.GaugeCronDuration.WithLabelValues("sync_alert_rules").Set(float64(ms))
|
||||
arc.stats.GaugeSyncNumber.WithLabelValues("sync_alert_rules").Set(float64(len(m)))
|
||||
logger.Infof("timer: sync rules done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("alert_rules", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -180,6 +180,7 @@ func (c *AlertSubscribeCacheType) syncAlertSubscribes() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
c.stats.GaugeCronDuration.WithLabelValues("sync_alert_subscribes").Set(float64(ms))
|
||||
c.stats.GaugeSyncNumber.WithLabelValues("sync_alert_subscribes").Set(float64(len(lst)))
|
||||
logger.Infof("timer: sync subscribes done, cost: %dms, number: %d", ms, len(lst))
|
||||
dumper.PutSyncRecord("alert_subscribes", start.Unix(), ms, len(lst), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -118,6 +118,8 @@ func (c *BusiGroupCacheType) syncBusiGroups() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
c.stats.GaugeCronDuration.WithLabelValues("sync_busi_groups").Set(float64(ms))
|
||||
c.stats.GaugeSyncNumber.WithLabelValues("sync_busi_groups").Set(float64(len(m)))
|
||||
|
||||
logger.Infof("timer: sync busi groups done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("busi_groups", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -86,6 +86,8 @@ func (c *ConfigCache) syncConfigs() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
c.stats.GaugeCronDuration.WithLabelValues("sync_user_variables").Set(float64(ms))
|
||||
c.stats.GaugeSyncNumber.WithLabelValues("sync_user_variables").Set(float64(len(decryptMap)))
|
||||
|
||||
logger.Infof("timer: sync user_variables done, cost: %dms, number: %d", ms, len(decryptMap))
|
||||
dumper.PutSyncRecord("user_variables", start.Unix(), ms, len(decryptMap), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -82,6 +82,8 @@ func (c *CvalCache) syncConfigs() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
c.stats.GaugeCronDuration.WithLabelValues("sync_cvals").Set(float64(ms))
|
||||
c.stats.GaugeSyncNumber.WithLabelValues("sync_cvals").Set(float64(len(c.cvals)))
|
||||
|
||||
logger.Infof("timer: sync cvals done, cost: %dms", ms)
|
||||
dumper.PutSyncRecord("cvals", start.Unix(), ms, len(c.cvals), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -134,6 +134,8 @@ func (d *DatasourceCacheType) syncDatasources() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
d.stats.GaugeCronDuration.WithLabelValues("sync_datasources").Set(float64(ms))
|
||||
d.stats.GaugeSyncNumber.WithLabelValues("sync_datasources").Set(float64(len(ds)))
|
||||
|
||||
logger.Infof("timer: sync datasources done, cost: %dms, number: %d", ms, len(ds))
|
||||
dumper.PutSyncRecord("datasources", start.Unix(), ms, len(ds), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -156,6 +156,7 @@ func (epc *EventProcessorCacheType) syncEventProcessors() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
epc.stats.GaugeCronDuration.WithLabelValues("sync_event_processors").Set(float64(ms))
|
||||
epc.stats.GaugeSyncNumber.WithLabelValues("sync_event_processors").Set(float64(len(m)))
|
||||
logger.Infof("timer: sync event processors done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("event_processors", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -132,6 +132,7 @@ func (mtc *MessageTemplateCacheType) syncMessageTemplates() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
mtc.stats.GaugeCronDuration.WithLabelValues("sync_message_templates").Set(float64(ms))
|
||||
mtc.stats.GaugeSyncNumber.WithLabelValues("sync_message_templates").Set(float64(len(m)))
|
||||
logger.Infof("timer: sync message templates done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("message_templates", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -2,10 +2,8 @@ package memsto
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -16,23 +14,9 @@ import (
|
||||
"github.com/ccfos/nightingale/v6/pkg/ctx"
|
||||
|
||||
"github.com/pkg/errors"
|
||||
"github.com/toolkits/pkg/container/list"
|
||||
"github.com/toolkits/pkg/logger"
|
||||
)
|
||||
|
||||
// NotifyTask 表示一个通知发送任务
|
||||
type NotifyTask struct {
|
||||
Events []*models.AlertCurEvent
|
||||
NotifyRuleId int64
|
||||
NotifyChannel *models.NotifyChannelConfig
|
||||
TplContent map[string]interface{}
|
||||
CustomParams map[string]string
|
||||
Sendtos []string
|
||||
}
|
||||
|
||||
// NotifyRecordFunc 通知记录函数类型
|
||||
type NotifyRecordFunc func(ctx *ctx.Context, events []*models.AlertCurEvent, notifyRuleId int64, channelName, target, resp string, err error)
|
||||
|
||||
type NotifyChannelCacheType struct {
|
||||
statTotal int64
|
||||
statLastUpdated int64
|
||||
@@ -40,18 +24,13 @@ type NotifyChannelCacheType struct {
|
||||
stats *Stats
|
||||
|
||||
sync.RWMutex
|
||||
channels map[int64]*models.NotifyChannelConfig // key: channel id
|
||||
channelsQueue map[int64]*list.SafeListLimited
|
||||
channels map[int64]*models.NotifyChannelConfig // key: channel id
|
||||
|
||||
httpConcurrency map[int64]chan struct{}
|
||||
|
||||
httpClient map[int64]*http.Client
|
||||
smtpCh map[int64]chan *models.EmailContext
|
||||
smtpQuitCh map[int64]chan struct{}
|
||||
|
||||
// 队列消费者控制
|
||||
queueQuitCh map[int64]chan struct{}
|
||||
|
||||
// 通知记录回调函数
|
||||
notifyRecordFunc NotifyRecordFunc
|
||||
}
|
||||
|
||||
func NewNotifyChannelCache(ctx *ctx.Context, stats *Stats) *NotifyChannelCacheType {
|
||||
@@ -61,20 +40,18 @@ func NewNotifyChannelCache(ctx *ctx.Context, stats *Stats) *NotifyChannelCacheTy
|
||||
ctx: ctx,
|
||||
stats: stats,
|
||||
channels: make(map[int64]*models.NotifyChannelConfig),
|
||||
channelsQueue: make(map[int64]*list.SafeListLimited),
|
||||
queueQuitCh: make(map[int64]chan struct{}),
|
||||
httpClient: make(map[int64]*http.Client),
|
||||
smtpCh: make(map[int64]chan *models.EmailContext),
|
||||
smtpQuitCh: make(map[int64]chan struct{}),
|
||||
}
|
||||
|
||||
ncc.SyncNotifyChannels()
|
||||
return ncc
|
||||
}
|
||||
|
||||
// SetNotifyRecordFunc 设置通知记录回调函数
|
||||
func (ncc *NotifyChannelCacheType) SetNotifyRecordFunc(fn NotifyRecordFunc) {
|
||||
ncc.notifyRecordFunc = fn
|
||||
func (ncc *NotifyChannelCacheType) Reset() {
|
||||
ncc.Lock()
|
||||
defer ncc.Unlock()
|
||||
|
||||
ncc.statTotal = -1
|
||||
ncc.statLastUpdated = -1
|
||||
ncc.channels = make(map[int64]*models.NotifyChannelConfig)
|
||||
}
|
||||
|
||||
func (ncc *NotifyChannelCacheType) StatChanged(total, lastUpdated int64) bool {
|
||||
@@ -85,257 +62,30 @@ func (ncc *NotifyChannelCacheType) StatChanged(total, lastUpdated int64) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
func (ncc *NotifyChannelCacheType) Set(m map[int64]*models.NotifyChannelConfig, total, lastUpdated int64) {
|
||||
func (ncc *NotifyChannelCacheType) Set(m map[int64]*models.NotifyChannelConfig, httpConcurrency map[int64]chan struct{}, httpClient map[int64]*http.Client,
|
||||
smtpCh map[int64]chan *models.EmailContext, quitCh map[int64]chan struct{}, total, lastUpdated int64) {
|
||||
ncc.Lock()
|
||||
defer ncc.Unlock()
|
||||
for _, k := range ncc.httpConcurrency {
|
||||
close(k)
|
||||
}
|
||||
ncc.httpConcurrency = httpConcurrency
|
||||
ncc.channels = m
|
||||
ncc.httpClient = httpClient
|
||||
ncc.smtpCh = smtpCh
|
||||
|
||||
// 1. 处理需要删除的通道
|
||||
ncc.removeDeletedChannels(m)
|
||||
for i := range ncc.smtpQuitCh {
|
||||
close(ncc.smtpQuitCh[i])
|
||||
}
|
||||
|
||||
// 2. 处理新增和更新的通道
|
||||
ncc.addOrUpdateChannels(m)
|
||||
ncc.smtpQuitCh = quitCh
|
||||
|
||||
ncc.Unlock()
|
||||
|
||||
// only one goroutine used, so no need lock
|
||||
ncc.statTotal = total
|
||||
ncc.statLastUpdated = lastUpdated
|
||||
}
|
||||
|
||||
// removeDeletedChannels 移除已删除的通道
|
||||
func (ncc *NotifyChannelCacheType) removeDeletedChannels(newChannels map[int64]*models.NotifyChannelConfig) {
|
||||
for chID := range ncc.channels {
|
||||
if _, exists := newChannels[chID]; !exists {
|
||||
logger.Infof("removing deleted channel %d", chID)
|
||||
|
||||
// 停止消费者协程
|
||||
if quitCh, exists := ncc.queueQuitCh[chID]; exists {
|
||||
close(quitCh)
|
||||
delete(ncc.queueQuitCh, chID)
|
||||
}
|
||||
|
||||
// 删除队列
|
||||
delete(ncc.channelsQueue, chID)
|
||||
|
||||
// 删除HTTP客户端
|
||||
delete(ncc.httpClient, chID)
|
||||
|
||||
// 停止SMTP发送器
|
||||
if quitCh, exists := ncc.smtpQuitCh[chID]; exists {
|
||||
close(quitCh)
|
||||
delete(ncc.smtpQuitCh, chID)
|
||||
delete(ncc.smtpCh, chID)
|
||||
}
|
||||
|
||||
// 删除通道配置
|
||||
delete(ncc.channels, chID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// addOrUpdateChannels 添加或更新通道
|
||||
func (ncc *NotifyChannelCacheType) addOrUpdateChannels(newChannels map[int64]*models.NotifyChannelConfig) {
|
||||
for chID, newChannel := range newChannels {
|
||||
oldChannel, exists := ncc.channels[chID]
|
||||
if exists {
|
||||
if ncc.channelConfigChanged(oldChannel, newChannel) {
|
||||
logger.Infof("updating channel %d (new: %t)", chID, !exists)
|
||||
ncc.stopChannelResources(chID)
|
||||
} else {
|
||||
logger.Infof("channel %d config not changed", chID)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
// 更新通道配置
|
||||
ncc.channels[chID] = newChannel
|
||||
|
||||
// 根据类型创建相应的资源
|
||||
switch newChannel.RequestType {
|
||||
case "http", "flashduty":
|
||||
// 创建HTTP客户端
|
||||
if newChannel.RequestConfig != nil && newChannel.RequestConfig.HTTPRequestConfig != nil {
|
||||
cli, err := models.GetHTTPClient(newChannel)
|
||||
if err != nil {
|
||||
logger.Warningf("failed to create HTTP client for channel %d: %v", chID, err)
|
||||
} else {
|
||||
if ncc.httpClient == nil {
|
||||
ncc.httpClient = make(map[int64]*http.Client)
|
||||
}
|
||||
ncc.httpClient[chID] = cli
|
||||
}
|
||||
}
|
||||
|
||||
// 对于 http 类型，启动队列和消费者
|
||||
if newChannel.RequestType == "http" {
|
||||
ncc.startHttpChannel(chID, newChannel)
|
||||
}
|
||||
case "smtp":
|
||||
// 创建SMTP发送器
|
||||
if newChannel.RequestConfig != nil && newChannel.RequestConfig.SMTPRequestConfig != nil {
|
||||
ch := make(chan *models.EmailContext)
|
||||
quit := make(chan struct{})
|
||||
go ncc.startEmailSender(chID, newChannel.RequestConfig.SMTPRequestConfig, ch, quit)
|
||||
|
||||
if ncc.smtpCh == nil {
|
||||
ncc.smtpCh = make(map[int64]chan *models.EmailContext)
|
||||
}
|
||||
if ncc.smtpQuitCh == nil {
|
||||
ncc.smtpQuitCh = make(map[int64]chan struct{})
|
||||
}
|
||||
ncc.smtpCh[chID] = ch
|
||||
ncc.smtpQuitCh[chID] = quit
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// channelConfigChanged 检查通道配置是否发生变化
|
||||
func (ncc *NotifyChannelCacheType) channelConfigChanged(oldChannel, newChannel *models.NotifyChannelConfig) bool {
|
||||
if oldChannel == nil || newChannel == nil {
|
||||
return true
|
||||
}
|
||||
|
||||
// check updateat
|
||||
if oldChannel.UpdateAt != newChannel.UpdateAt {
|
||||
return true
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// stopChannelResources 停止通道的相关资源
|
||||
func (ncc *NotifyChannelCacheType) stopChannelResources(chID int64) {
|
||||
// 停止HTTP消费者协程
|
||||
if quitCh, exists := ncc.queueQuitCh[chID]; exists {
|
||||
close(quitCh)
|
||||
delete(ncc.queueQuitCh, chID)
|
||||
delete(ncc.channelsQueue, chID)
|
||||
}
|
||||
|
||||
// 停止SMTP发送器
|
||||
if quitCh, exists := ncc.smtpQuitCh[chID]; exists {
|
||||
close(quitCh)
|
||||
delete(ncc.smtpQuitCh, chID)
|
||||
delete(ncc.smtpCh, chID)
|
||||
}
|
||||
}
|
||||
|
||||
// startHttpChannel 启动HTTP通道的队列和消费者
|
||||
func (ncc *NotifyChannelCacheType) startHttpChannel(chID int64, channel *models.NotifyChannelConfig) {
|
||||
if channel.RequestConfig == nil || channel.RequestConfig.HTTPRequestConfig == nil {
|
||||
logger.Warningf("notify channel %+v http request config not found", channel)
|
||||
return
|
||||
}
|
||||
|
||||
// 创建队列
|
||||
queue := list.NewSafeListLimited(100000)
|
||||
ncc.channelsQueue[chID] = queue
|
||||
|
||||
// 启动消费者协程
|
||||
quitCh := make(chan struct{})
|
||||
ncc.queueQuitCh[chID] = quitCh
|
||||
|
||||
// 启动指定数量的消费者协程
|
||||
concurrency := channel.RequestConfig.HTTPRequestConfig.Concurrency
|
||||
for i := 0; i < concurrency; i++ {
|
||||
go ncc.startNotifyConsumer(chID, queue, quitCh)
|
||||
}
|
||||
|
||||
logger.Infof("started %d notify consumers for channel %d", concurrency, chID)
|
||||
}
|
||||
|
||||
// 启动通知消费者协程
|
||||
func (ncc *NotifyChannelCacheType) startNotifyConsumer(channelID int64, queue *list.SafeListLimited, quitCh chan struct{}) {
|
||||
logger.Infof("starting notify consumer for channel %d", channelID)
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-quitCh:
|
||||
logger.Infof("notify consumer for channel %d stopped", channelID)
|
||||
return
|
||||
default:
|
||||
// 从队列中取出任务
|
||||
task := queue.PopBack()
|
||||
if task == nil {
|
||||
// 队列为空，等待一段时间
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
continue
|
||||
}
|
||||
|
||||
notifyTask, ok := task.(*NotifyTask)
|
||||
if !ok {
|
||||
logger.Errorf("invalid task type in queue for channel %d", channelID)
|
||||
continue
|
||||
}
|
||||
|
||||
// 处理通知任务
|
||||
ncc.processNotifyTask(notifyTask)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// processNotifyTask 处理通知任务（仅处理 http 类型）
|
||||
func (ncc *NotifyChannelCacheType) processNotifyTask(task *NotifyTask) {
|
||||
httpClient := ncc.GetHttpClient(task.NotifyChannel.ID)
|
||||
|
||||
// 现在只处理 http 类型，flashduty 保持直接发送
|
||||
if task.NotifyChannel.RequestType == "http" {
|
||||
if len(task.Sendtos) == 0 || ncc.needBatchContacts(task.NotifyChannel.RequestConfig.HTTPRequestConfig) {
|
||||
start := time.Now()
|
||||
resp, err := task.NotifyChannel.SendHTTP(task.Events, task.TplContent, task.CustomParams, task.Sendtos, httpClient)
|
||||
resp = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), resp)
|
||||
logger.Infof("notify_id: %d, channel_name: %v, event:%+v, tplContent:%v, customParams:%v, userInfo:%+v, respBody: %v, err: %v",
|
||||
task.NotifyRuleId, task.NotifyChannel.Name, task.Events[0], task.TplContent, task.CustomParams, task.Sendtos, resp, err)
|
||||
|
||||
// 调用通知记录回调函数
|
||||
if ncc.notifyRecordFunc != nil {
|
||||
ncc.notifyRecordFunc(ncc.ctx, task.Events, task.NotifyRuleId, task.NotifyChannel.Name, ncc.getSendTarget(task.CustomParams, task.Sendtos), resp, err)
|
||||
}
|
||||
} else {
|
||||
for i := range task.Sendtos {
|
||||
start := time.Now()
|
||||
resp, err := task.NotifyChannel.SendHTTP(task.Events, task.TplContent, task.CustomParams, []string{task.Sendtos[i]}, httpClient)
|
||||
resp = fmt.Sprintf("duration: %d ms %s", time.Since(start).Milliseconds(), resp)
|
||||
logger.Infof("notify_id: %d, channel_name: %v, event:%+v, tplContent:%v, customParams:%v, userInfo:%+v, respBody: %v, err: %v",
|
||||
task.NotifyRuleId, task.NotifyChannel.Name, task.Events[0], task.TplContent, task.CustomParams, task.Sendtos[i], resp, err)
|
||||
|
||||
// 调用通知记录回调函数
|
||||
if ncc.notifyRecordFunc != nil {
|
||||
ncc.notifyRecordFunc(ncc.ctx, task.Events, task.NotifyRuleId, task.NotifyChannel.Name, ncc.getSendTarget(task.CustomParams, []string{task.Sendtos[i]}), resp, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 判断是否需要批量发送联系人
|
||||
func (ncc *NotifyChannelCacheType) needBatchContacts(requestConfig *models.HTTPRequestConfig) bool {
|
||||
if requestConfig == nil {
|
||||
return false
|
||||
}
|
||||
b, _ := json.Marshal(requestConfig)
|
||||
return strings.Contains(string(b), "$sendtos")
|
||||
}
|
||||
|
||||
// 获取发送目标
|
||||
func (ncc *NotifyChannelCacheType) getSendTarget(customParams map[string]string, sendtos []string) string {
|
||||
if len(customParams) == 0 {
|
||||
return strings.Join(sendtos, ",")
|
||||
}
|
||||
|
||||
values := make([]string, 0)
|
||||
for _, value := range customParams {
|
||||
runes := []rune(value)
|
||||
if len(runes) <= 4 {
|
||||
values = append(values, value)
|
||||
} else {
|
||||
maskedValue := string(runes[:len(runes)-4]) + "****"
|
||||
values = append(values, maskedValue)
|
||||
}
|
||||
}
|
||||
|
||||
return strings.Join(values, ",")
|
||||
}
|
||||
|
||||
func (ncc *NotifyChannelCacheType) Get(channelId int64) *models.NotifyChannelConfig {
|
||||
ncc.RLock()
|
||||
defer ncc.RUnlock()
|
||||
@@ -367,25 +117,6 @@ func (ncc *NotifyChannelCacheType) GetChannelIds() []int64 {
|
||||
return list
|
||||
}
|
||||
|
||||
// 新增：将通知任务加入队列
|
||||
func (ncc *NotifyChannelCacheType) EnqueueNotifyTask(task *NotifyTask) bool {
|
||||
ncc.RLock()
|
||||
queue := ncc.channelsQueue[task.NotifyChannel.ID]
|
||||
ncc.RUnlock()
|
||||
|
||||
if queue == nil {
|
||||
logger.Errorf("no queue found for channel %d", task.NotifyChannel.ID)
|
||||
return false
|
||||
}
|
||||
|
||||
success := queue.PushFront(task)
|
||||
if !success {
|
||||
logger.Warningf("failed to enqueue notify task for channel %d, queue is full", task.NotifyChannel.ID)
|
||||
}
|
||||
|
||||
return success
|
||||
}
|
||||
|
||||
func (ncc *NotifyChannelCacheType) SyncNotifyChannels() {
|
||||
err := ncc.syncNotifyChannels()
|
||||
if err != nil {
|
||||
@@ -431,12 +162,43 @@ func (ncc *NotifyChannelCacheType) syncNotifyChannels() error {
|
||||
m[lst[i].ID] = lst[i]
|
||||
}
|
||||
|
||||
// 增量更新：只传递通道配置，让增量更新逻辑按需创建资源
|
||||
ncc.Set(m, stat.Total, stat.LastUpdated)
|
||||
httpConcurrency := make(map[int64]chan struct{})
|
||||
httpClient := make(map[int64]*http.Client)
|
||||
smtpCh := make(map[int64]chan *models.EmailContext)
|
||||
quitCh := make(map[int64]chan struct{})
|
||||
|
||||
for i := range lst {
|
||||
// todo 优化变更粒度
|
||||
|
||||
switch lst[i].RequestType {
|
||||
case "http", "flashduty":
|
||||
if lst[i].RequestConfig == nil || lst[i].RequestConfig.HTTPRequestConfig == nil {
|
||||
logger.Warningf("notify channel %+v http request config not found", lst[i])
|
||||
continue
|
||||
}
|
||||
|
||||
cli, _ := models.GetHTTPClient(lst[i])
|
||||
httpClient[lst[i].ID] = cli
|
||||
httpConcurrency[lst[i].ID] = make(chan struct{}, lst[i].RequestConfig.HTTPRequestConfig.Concurrency)
|
||||
for j := 0; j < lst[i].RequestConfig.HTTPRequestConfig.Concurrency; j++ {
|
||||
httpConcurrency[lst[i].ID] <- struct{}{}
|
||||
}
|
||||
case "smtp":
|
||||
ch := make(chan *models.EmailContext)
|
||||
quit := make(chan struct{})
|
||||
go ncc.startEmailSender(lst[i].ID, lst[i].RequestConfig.SMTPRequestConfig, ch, quit)
|
||||
smtpCh[lst[i].ID] = ch
|
||||
quitCh[lst[i].ID] = quit
|
||||
default:
|
||||
}
|
||||
}
|
||||
|
||||
ncc.Set(m, httpConcurrency, httpClient, smtpCh, quitCh, stat.Total, stat.LastUpdated)
|
||||
|
||||
ms := time.Since(start).Milliseconds()
|
||||
ncc.stats.GaugeCronDuration.WithLabelValues("sync_notify_channels").Set(float64(ms))
|
||||
ncc.stats.GaugeSyncNumber.WithLabelValues("sync_notify_channels").Set(float64(len(m)))
|
||||
logger.Infof("timer: sync notify channels done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("notify_channels", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
@@ -543,3 +305,22 @@ func (ncc *NotifyChannelCacheType) dialSmtp(quitCh chan struct{}, d *gomail.Dial
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (ncc *NotifyChannelCacheType) HttpConcurrencyAdd(channelId int64) bool {
|
||||
ncc.RLock()
|
||||
defer ncc.RUnlock()
|
||||
if _, ok := ncc.httpConcurrency[channelId]; !ok {
|
||||
return false
|
||||
}
|
||||
_, ok := <-ncc.httpConcurrency[channelId]
|
||||
return ok
|
||||
}
|
||||
|
||||
func (ncc *NotifyChannelCacheType) HttpConcurrencyDone(channelId int64) {
|
||||
ncc.RLock()
|
||||
defer ncc.RUnlock()
|
||||
if _, ok := ncc.httpConcurrency[channelId]; !ok {
|
||||
return
|
||||
}
|
||||
ncc.httpConcurrency[channelId] <- struct{}{}
|
||||
}
|
||||
|
||||
@@ -132,6 +132,7 @@ func (nrc *NotifyRuleCacheType) syncNotifyRules() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
nrc.stats.GaugeCronDuration.WithLabelValues("sync_notify_rules").Set(float64(ms))
|
||||
nrc.stats.GaugeSyncNumber.WithLabelValues("sync_notify_rules").Set(float64(len(m)))
|
||||
logger.Infof("timer: sync notify rules done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("notify_rules", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -133,6 +133,7 @@ func (rrc *RecordingRuleCacheType) syncRecordingRules() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
rrc.stats.GaugeCronDuration.WithLabelValues("sync_recording_rules").Set(float64(ms))
|
||||
rrc.stats.GaugeSyncNumber.WithLabelValues("sync_recording_rules").Set(float64(len(m)))
|
||||
logger.Infof("timer: sync recording rules done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("recording_rules", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -179,6 +179,7 @@ func (tc *TargetCacheType) syncTargets() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
tc.stats.GaugeCronDuration.WithLabelValues("sync_targets").Set(float64(ms))
|
||||
tc.stats.GaugeSyncNumber.WithLabelValues("sync_targets").Set(float64(len(lst)))
|
||||
logger.Infof("timer: sync targets done, cost: %dms, number: %d", ms, len(lst))
|
||||
dumper.PutSyncRecord("targets", start.Unix(), ms, len(lst), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -84,6 +84,7 @@ func (ttc *TaskTplCache) syncTaskTpl() error {
|
||||
ttc.Set(m, stat.Total, stat.LastUpdated)
|
||||
|
||||
ms := time.Since(start).Milliseconds()
|
||||
logger.Infof("timer: sync task tpls done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("task_tpls", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -189,6 +189,8 @@ func (uc *UserCacheType) syncUsers() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
uc.stats.GaugeCronDuration.WithLabelValues("sync_users").Set(float64(ms))
|
||||
uc.stats.GaugeSyncNumber.WithLabelValues("sync_users").Set(float64(len(m)))
|
||||
|
||||
logger.Infof("timer: sync users done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("users", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -158,6 +158,8 @@ func (ugc *UserGroupCacheType) syncUserGroups() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
ugc.stats.GaugeCronDuration.WithLabelValues("sync_user_groups").Set(float64(ms))
|
||||
ugc.stats.GaugeSyncNumber.WithLabelValues("sync_user_groups").Set(float64(len(m)))
|
||||
|
||||
logger.Infof("timer: sync user groups done, cost: %dms, number: %d", ms, len(m))
|
||||
dumper.PutSyncRecord("user_groups", start.Unix(), ms, len(m), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -168,6 +168,8 @@ func (utc *UserTokenCacheType) syncUserTokens() error {
|
||||
ms := time.Since(start).Milliseconds()
|
||||
utc.stats.GaugeCronDuration.WithLabelValues("sync_user_tokens").Set(float64(ms))
|
||||
utc.stats.GaugeSyncNumber.WithLabelValues("sync_user_tokens").Set(float64(len(tokenUsers)))
|
||||
|
||||
logger.Infof("timer: sync user tokens done, cost: %dms, number: %d", ms, len(tokenUsers))
|
||||
dumper.PutSyncRecord("user_tokens", start.Unix(), ms, len(tokenUsers), "success")
|
||||
|
||||
return nil
|
||||
|
||||
@@ -30,8 +30,6 @@ const (
|
||||
ELASTICSEARCH = "elasticsearch"
|
||||
MYSQL = "mysql"
|
||||
POSTGRESQL = "pgsql"
|
||||
DORIS = "doris"
|
||||
OPENSEARCH = "opensearch"
|
||||
|
||||
CLICKHOUSE = "ck"
|
||||
)
|
||||
@@ -1201,9 +1199,7 @@ func (ar *AlertRule) IsInnerRule() bool {
|
||||
ar.Cate == ELASTICSEARCH ||
|
||||
ar.Prod == LOKI || ar.Cate == LOKI ||
|
||||
ar.Cate == MYSQL ||
|
||||
ar.Cate == POSTGRESQL ||
|
||||
ar.Cate == DORIS ||
|
||||
ar.Cate == OPENSEARCH
|
||||
ar.Cate == POSTGRESQL
|
||||
}
|
||||
|
||||
func (ar *AlertRule) GetRuleType() string {
|
||||
|
||||
@@ -132,10 +132,6 @@ func (s *AlertSubscribe) Verify() error {
|
||||
}
|
||||
}
|
||||
|
||||
if s.NotifyVersion == 1 && len(s.NotifyRuleIds) == 0 {
|
||||
return errors.New("no notify rules selected")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
@@ -242,12 +242,7 @@ var NewTplMap = map[string]string{
|
||||
- {{$key}}: {{$val}}
|
||||
{{- end}}
|
||||
{{- end}}
|
||||
{{if $event.AnnotationsJSON}}
|
||||
- **附加信息**:
|
||||
{{- range $key, $val := $event.AnnotationsJSON}}
|
||||
- {{$key}}: {{$val}}
|
||||
{{- end}}
|
||||
{{end}}
|
||||
|
||||
{{$domain := "http://127.0.0.1:17000" }}
|
||||
{{$mutelink := print $domain "/alert-mutes/add?busiGroup=" $event.GroupId "&cate=" $event.Cate "&datasource_ids=" $event.DatasourceId "&prod=" $event.RuleProd}}
|
||||
{{- range $key, $value := $event.TagsMap}}
|
||||
@@ -476,10 +471,6 @@ var NewTplMap = map[string]string{
|
||||
规则名称: {{$event.RuleName}}{{if $event.RuleNote}}
|
||||
规则备注: {{$event.RuleNote}}{{end}}
|
||||
监控指标: {{$event.TagsJSON}}
|
||||
附加信息:
|
||||
{{- range $key, $val := $event.AnnotationsJSON}}
|
||||
{{$key}}: {{$val}}
|
||||
{{- end}}
|
||||
{{if $event.IsRecovered}}恢复时间：{{timeformat $event.LastEvalTime}}{{else}}触发时间: {{timeformat $event.TriggerTime}}
|
||||
触发时值: {{$event.TriggerValue}}{{end}}
|
||||
发送时间: {{timestamp}}{{$domain := "http://127.0.0.1:17000" }}
|
||||
@@ -501,15 +492,9 @@ var NewTplMap = map[string]string{
|
||||
**事件标签:** {{$event.TagsJSON}}
|
||||
**触发时间:** {{timeformat $event.TriggerTime}}
|
||||
**发送时间:** {{timestamp}}
|
||||
**触发时值:** {{$event.TriggerValue}}
|
||||
**触发时值:** {{$event.TriggerValue}}
|
||||
{{if $event.RuleNote }}**告警描述:** **{{$event.RuleNote}}**{{end}}
|
||||
{{- end -}}
|
||||
{{if $event.AnnotationsJSON}}
|
||||
**附加信息**:
|
||||
{{- range $key, $val := $event.AnnotationsJSON}}
|
||||
{{$key}}: {{$val}}
|
||||
{{- end}}
|
||||
{{- end}}
|
||||
{{$domain := "http://请联系管理员修改通知模板将域名替换为实际的域名" }}
|
||||
[事件详情]({{$domain}}/alert-his-events/{{$event.Id}})|[屏蔽1小时]({{$domain}}/alert-mutes/add?busiGroup={{$event.GroupId}}&cate={{$event.Cate}}&datasource_ids={{$event.DatasourceId}}&prod={{$event.RuleProd}}{{range $key, $value := $event.TagsMap}}&tags={{$key}}%3D{{$value}}{{end}})|[查看曲线]({{$domain}}/metric/explorer?data_source_id={{$event.DatasourceId}}&data_source_name=prometheus&mode=graph&prom_ql={{$event.PromQl|escape}})`,
|
||||
EmailSubject: `{{if $event.IsRecovered}}Recovered{{else}}Triggered{{end}}: {{$event.RuleName}} {{$event.TagsJSON}}`,
|
||||
@@ -533,8 +518,7 @@ var NewTplMap = map[string]string{
|
||||
**规则标题**: {{$event.RuleName}}{{if $event.RuleNote}}
|
||||
**规则备注**: {{$event.RuleNote}}{{end}}{{if $event.TargetIdent}}
|
||||
**监控对象**: {{$event.TargetIdent}}{{end}}
|
||||
**监控指标**: {{$event.TagsJSON}}
|
||||
{{if $event.AnnotationsJSON}}**附加信息**:{{range $key, $val := $event.AnnotationsJSON}}{{$key}}:{{$val}} {{end}} {{end}}{{if not $event.IsRecovered}}
|
||||
**监控指标**: {{$event.TagsJSON}}{{if not $event.IsRecovered}}
|
||||
**触发时值**: {{$event.TriggerValue}}{{end}}
|
||||
{{if $event.IsRecovered}}**恢复时间**: {{timeformat $event.LastEvalTime}}{{else}}**首次触发时间**: {{timeformat $event.FirstTriggerTime}}{{end}}
|
||||
{{$time_duration := sub now.Unix $event.FirstTriggerTime }}{{if $event.IsRecovered}}{{$time_duration = sub $event.LastEvalTime $event.FirstTriggerTime }}{{end}}**距离首次告警**: {{humanizeDurationInterface $time_duration}}
|
||||
|
||||
102
pkg/ginx/auth.go
102
pkg/ginx/auth.go
@@ -1,102 +0,0 @@
|
||||
// Copyright 2014 Manu Martinez-Almeida. All rights reserved.
|
||||
// Use of this source code is governed by a MIT style
|
||||
// license that can be found in the LICENSE file.
|
||||
|
||||
package ginx
|
||||
|
||||
import (
|
||||
"crypto/subtle"
|
||||
"encoding/base64"
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
// AuthUserKey is the cookie name for user credential in basic auth.
|
||||
const AuthUserKey = "user"
|
||||
|
||||
// Accounts defines a key/value for user/pass list of authorized logins.
|
||||
type Accounts []Account
|
||||
|
||||
type Account struct {
|
||||
User string
|
||||
Password string
|
||||
}
|
||||
|
||||
type authPair struct {
|
||||
value string
|
||||
user string
|
||||
}
|
||||
|
||||
type authPairs []authPair
|
||||
|
||||
func (a authPairs) searchCredential(authValue string) (string, bool) {
|
||||
if authValue == "" {
|
||||
return "", false
|
||||
}
|
||||
for _, pair := range a {
|
||||
if subtle.ConstantTimeCompare(StringToBytes(pair.value), StringToBytes(authValue)) == 1 {
|
||||
return pair.user, true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// BasicAuthForRealm returns a Basic HTTP Authorization middleware. It takes as arguments a map[string]string where
|
||||
// the key is the user name and the value is the password, as well as the name of the Realm.
|
||||
// If the realm is empty, "Authorization Required" will be used by default.
|
||||
// (see http://tools.ietf.org/html/rfc2617#section-1.2)
|
||||
func BasicAuthForRealm(accounts Accounts, realm string) gin.HandlerFunc {
|
||||
if realm == "" {
|
||||
realm = "Authorization Required"
|
||||
}
|
||||
realm = "Basic realm=" + strconv.Quote(realm)
|
||||
pairs := processAccounts(accounts)
|
||||
return func(c *gin.Context) {
|
||||
// Search user in the slice of allowed credentials
|
||||
user, found := pairs.searchCredential(c.Request.Header.Get("Authorization"))
|
||||
if !found {
|
||||
// Credentials doesn't match, we return 401 and abort handlers chain.
|
||||
c.Header("WWW-Authenticate", realm)
|
||||
c.AbortWithStatus(http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
// The user credentials was found, set user's id to key AuthUserKey in this context, the user's id can be read later using
|
||||
// c.MustGet(gin.AuthUserKey).
|
||||
c.Set(AuthUserKey, user)
|
||||
}
|
||||
}
|
||||
|
||||
// BasicAuth returns a Basic HTTP Authorization middleware. It takes as argument a map[string]string where
|
||||
// the key is the user name and the value is the password.
|
||||
func BasicAuth(accounts Accounts) gin.HandlerFunc {
|
||||
return BasicAuthForRealm(accounts, "")
|
||||
}
|
||||
|
||||
func processAccounts(accounts Accounts) authPairs {
|
||||
length := len(accounts)
|
||||
assert1(length > 0, "Empty list of authorized credentials")
|
||||
pairs := make(authPairs, 0, length)
|
||||
for _, account := range accounts {
|
||||
assert1(account.User != "", "User can not be empty")
|
||||
value := authorizationHeader(account.User, account.Password)
|
||||
pairs = append(pairs, authPair{
|
||||
value: value,
|
||||
user: account.User,
|
||||
})
|
||||
}
|
||||
return pairs
|
||||
}
|
||||
|
||||
func authorizationHeader(user, password string) string {
|
||||
base := user + ":" + password
|
||||
return "Basic " + base64.StdEncoding.EncodeToString(StringToBytes(base))
|
||||
}
|
||||
|
||||
func assert1(guard bool, text string) {
|
||||
if !guard {
|
||||
panic(text)
|
||||
}
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
// Copyright 2023 Gin Core Team. All rights reserved.
|
||||
// Use of this source code is governed by a MIT style
|
||||
// license that can be found in the LICENSE file.
|
||||
|
||||
//go:build go1.20
|
||||
|
||||
package ginx
|
||||
|
||||
import (
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
// StringToBytes converts string to byte slice without a memory allocation.
|
||||
// For more details, see https://github.com/golang/go/issues/53003#issuecomment-1140276077.
|
||||
func StringToBytes(s string) []byte {
|
||||
return unsafe.Slice(unsafe.StringData(s), len(s))
|
||||
}
|
||||
|
||||
// BytesToString converts byte slice to string without a memory allocation.
|
||||
// For more details, see https://github.com/golang/go/issues/53003#issuecomment-1140276077.
|
||||
func BytesToString(b []byte) string {
|
||||
return unsafe.String(unsafe.SliceData(b), len(b))
|
||||
}
|
||||
@@ -51,26 +51,6 @@ var I18N = `{
|
||||
"builtin payload already exists": "内置模板已存在",
|
||||
"This functionality has not been enabled. Please contact the system administrator to activate it.": "此功能尚未启用。请联系系统管理员启用",
|
||||
"targets not exist: %s": "有些机器不存在: %s",
|
||||
"mute is disabled": "屏蔽规则已禁用",
|
||||
"datasource id not match": "数据源ID不匹配",
|
||||
"event trigger time not within mute time range": "事件触发时间不在屏蔽时间范围内",
|
||||
"event trigger time not within periodic mute range": "事件触发时间不在周期性屏蔽时间范围内",
|
||||
"mute time type invalid": "屏蔽时间类型无效",
|
||||
"event severity not match mute severity": "事件严重程度与屏蔽严重程度不匹配",
|
||||
"event tags not match mute tags": "事件标签与屏蔽标签不匹配",
|
||||
"event datasource not match": "事件数据源不匹配",
|
||||
"event rule id not match": "事件告警规则ID不匹配",
|
||||
"event tags not match": "事件标签不匹配",
|
||||
"event group name not match": "事件业务组名称不匹配",
|
||||
"event severity not match": "事件严重程度不匹配",
|
||||
"subscribe notify rule not found: %v": "订阅通知规则未找到: %v",
|
||||
"notify rule send error: %v": "通知规则发送错误: %v",
|
||||
"event match subscribe and notification test ok": "事件匹配订阅规则，通知测试成功",
|
||||
"no notify rules selected": "未选择通知规则",
|
||||
"no notify channels selected": "未选择通知渠道",
|
||||
"no notify groups selected": "未选择通知组",
|
||||
"all users missing notify channel configurations: %v": "所有用户缺少通知渠道配置: %v",
|
||||
"event match subscribe and notify settings ok": "事件匹配订阅规则，通知设置正常",
|
||||
|
||||
"Infrastructure": "基础设施",
|
||||
"Host - View": "机器 - 查看",
|
||||
@@ -235,26 +215,6 @@ var I18N = `{
|
||||
"AlertRule already exists": "告警規則已存在",
|
||||
"This functionality has not been enabled. Please contact the system administrator to activate it.": "此功能尚未啟用。請聯繫系統管理員啟用",
|
||||
"targets not exist: %s": "有些機器不存在: %s",
|
||||
"mute is disabled": "屏蔽規則已禁用",
|
||||
"datasource id not match": "數據源ID不匹配",
|
||||
"event trigger time not within mute time range": "事件觸發時間不在屏蔽時間範圍內",
|
||||
"event trigger time not within periodic mute range": "事件觸發時間不在週期性屏蔽時間範圍內",
|
||||
"mute time type invalid": "屏蔽時間類型無效",
|
||||
"event severity not match mute severity": "事件嚴重程度與屏蔽嚴重程度不匹配",
|
||||
"event tags not match mute tags": "事件標籤與屏蔽標籤不匹配",
|
||||
"event datasource not match": "事件數據源不匹配",
|
||||
"event rule id not match": "事件告警規則ID不匹配",
|
||||
"event tags not match": "事件標籤不匹配",
|
||||
"event group name not match": "事件業務組名稱不匹配",
|
||||
"event severity not match": "事件嚴重程度不匹配",
|
||||
"subscribe notify rule not found: %v": "訂閱通知規則未找到: %v",
|
||||
"notify rule send error: %v": "通知規則發送錯誤: %v",
|
||||
"event match subscribe and notification test ok": "事件匹配訂閱規則，通知測試成功",
|
||||
"no notify rules selected": "未選擇通知規則",
|
||||
"no notify channels selected": "未選擇通知渠道",
|
||||
"no notify groups selected": "未選擇通知組",
|
||||
"all users missing notify channel configurations: %v": "所有用戶缺少通知渠道配置: %v",
|
||||
"event match subscribe and notify settings ok": "事件匹配訂閱規則，通知設置正常",
|
||||
|
||||
"Infrastructure": "基礎設施",
|
||||
"Host - View": "機器 - 查看",
|
||||
@@ -416,26 +376,6 @@ var I18N = `{
|
||||
"builtin payload already exists": "ビルトインテンプレートは既に存在します",
|
||||
"This functionality has not been enabled. Please contact the system administrator to activate it.": "この機能はまだ有効になっていません。システム管理者に連絡して有効にしてください",
|
||||
"targets not exist: %s": "いくつかのマシンが存在しません: %s",
|
||||
"mute is disabled": "ミュートルールが無効になっています",
|
||||
"datasource id not match": "データソースIDが一致しません",
|
||||
"event trigger time not within mute time range": "イベントトリガー時間がミュート時間範囲内にありません",
|
||||
"event trigger time not within periodic mute range": "イベントトリガー時間が周期的ミュート時間範囲内にありません",
|
||||
"mute time type invalid": "ミュート時間タイプが無効です",
|
||||
"event severity not match mute severity": "イベントの重要度がミュートの重要度と一致しません",
|
||||
"event tags not match mute tags": "イベントタグがミュートタグと一致しません",
|
||||
"event datasource not match": "イベントデータソースが一致しません",
|
||||
"event rule id not match": "イベントアラートルールIDが一致しません",
|
||||
"event tags not match": "イベントタグが一致しません",
|
||||
"event group name not match": "イベントビジネスグループ名が一致しません",
|
||||
"event severity not match": "イベントの重要度が一致しません",
|
||||
"subscribe notify rule not found: %v": "サブスクライブ通知ルールが見つかりません: %v",
|
||||
"notify rule send error: %v": "通知ルール送信エラー: %v",
|
||||
"event match subscribe and notification test ok": "イベントがサブスクライブルールに一致し、通知テストが成功しました",
|
||||
"no notify rules selected": "通知ルールが選択されていません",
|
||||
"no notify channels selected": "通知チャンネルが選択されていません",
|
||||
"no notify groups selected": "通知グループが選択されていません",
|
||||
"all users missing notify channel configurations: %v": "すべてのユーザーに通知チャンネル設定がありません: %v",
|
||||
"event match subscribe and notify settings ok": "イベントがサブスクライブルールに一致し、通知設定が正常です",
|
||||
|
||||
"Infrastructure": "インフラストラクチャ",
|
||||
"Host - View": "機器 - 閲覧",
|
||||
@@ -597,26 +537,6 @@ var I18N = `{
|
||||
"builtin payload already exists": "Встроенный шаблон уже существует",
|
||||
"This functionality has not been enabled. Please contact the system administrator to activate it.": "Эта функция не активирована. Пожалуйста, обратитесь к системному администратору для активации",
|
||||
"targets not exist: %s": "Некоторые машины не существуют: %s",
|
||||
"mute is disabled": "Правило отключения оповещений деактивировано",
|
||||
"datasource id not match": "Идентификатор источника данных не совпадает",
|
||||
"event trigger time not within mute time range": "Время срабатывания события не входит в диапазон времени отключения оповещений",
|
||||
"event trigger time not within periodic mute range": "Время срабатывания события не входит в периодический диапазон отключения оповещений",
|
||||
"mute time type invalid": "Недопустимый тип времени отключения оповещений",
|
||||
"event severity not match mute severity": "Уровень важности события не соответствует уровню важности отключения оповещений",
|
||||
"event tags not match mute tags": "Теги события не соответствуют тегам отключения оповещений",
|
||||
"event datasource not match": "Источник данных события не соответствует",
|
||||
"event rule id not match": "Идентификатор правила оповещения события не соответствует",
|
||||
"event tags not match": "Теги события не соответствуют",
|
||||
"event group name not match": "Название бизнес-группы события не соответствует",
|
||||
"event severity not match": "Уровень важности события не соответствует",
|
||||
"subscribe notify rule not found: %v": "Правило уведомления подписки не найдено: %v",
|
||||
"notify rule send error: %v": "Ошибка отправки правила уведомления: %v",
|
||||
"event match subscribe and notification test ok": "Событие соответствует правилу подписки, тест уведомления успешен",
|
||||
"no notify rules selected": "Правила уведомлений не выбраны",
|
||||
"no notify channels selected": "Каналы уведомлений не выбраны",
|
||||
"no notify groups selected": "Группы уведомлений не выбраны",
|
||||
"all users missing notify channel configurations: %v": "У всех пользователей отсутствуют настройки каналов уведомлений: %v",
|
||||
"event match subscribe and notify settings ok": "Событие соответствует правилу подписки, настройки уведомлений в порядке",
|
||||
|
||||
"Infrastructure": "Инфраструктура",
|
||||
"Host - View": "Хост - Просмотр",
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"github.com/ccfos/nightingale/v6/center/metas"
|
||||
"github.com/ccfos/nightingale/v6/memsto"
|
||||
"github.com/ccfos/nightingale/v6/pkg/ctx"
|
||||
"github.com/ccfos/nightingale/v6/pkg/ginx"
|
||||
"github.com/ccfos/nightingale/v6/pkg/httpx"
|
||||
"github.com/ccfos/nightingale/v6/pushgw/idents"
|
||||
"github.com/ccfos/nightingale/v6/pushgw/pconf"
|
||||
@@ -87,22 +86,15 @@ func (rt *Router) Config(r *gin.Engine) {
|
||||
|
||||
if len(rt.HTTP.APIForAgent.BasicAuth) > 0 {
|
||||
// enable basic auth
|
||||
accounts := make(ginx.Accounts, 0)
|
||||
accounts := make(gin.Accounts)
|
||||
for username, password := range rt.HTTP.APIForAgent.BasicAuth {
|
||||
accounts = append(accounts, ginx.Account{
|
||||
User: username,
|
||||
Password: password,
|
||||
})
|
||||
accounts[username] = password
|
||||
}
|
||||
|
||||
for username, password := range rt.HTTP.APIForService.BasicAuth {
|
||||
accounts = append(accounts, ginx.Account{
|
||||
User: username,
|
||||
Password: password,
|
||||
})
|
||||
accounts[username] = password
|
||||
}
|
||||
|
||||
auth := ginx.BasicAuth(accounts)
|
||||
auth := gin.BasicAuth(accounts)
|
||||
r.POST("/opentsdb/put", auth, rt.openTSDBPut)
|
||||
r.POST("/openfalcon/push", auth, rt.falconPush)
|
||||
r.POST("/prometheus/v1/write", auth, rt.remoteWrite)
|
||||
|
||||
Reference in New Issue
Block a user