"Add Prometheus Support with Documentation and Docker Compose Integration to OpenIM" (#1403)

* Code adaptation k8s: service discovery and registration adaptation, configuration adaptation

* Initial submission of the help charts script for openim API

* change the help charts script

* change the help charts script

* change helm chart codes

* change dockerfiles script

* change chart script:add configmap mounts

* change chart script:change repository

* change chart script:msggateway add one service

* change config.yaml

* roll back some config values

* change chart script:change Ingress rule with a rewrite annotation

* add mysql charts scrible

* change chart script:add mysql.config.yaml

* add nfs provisioner charts

* change chart script:add nfs.config.yaml

* add ingress-nginx charts

* change chart script:add ingress-nginx.config.yaml

* add redis &mongodb charts

* add kafka&minio charts

* change chart script:change redis.values.yaml

* change chart script:add redis.config.yaml

* change chart script:change redis.config.yaml

* change chart script:change mongodb.value.yaml

* change chart script:change mongodb.value.yaml

* change chart script:add mongodb.config.yaml

* change chart script:change minio.values.yaml

* change chart script:add minio.config.yaml

* change chart script:change kafka.values.yaml

* change chart script:add kafka.config.yaml

* change chart script:change services.config.yaml

* bug fix:Delete websocket's Port restrictions

* bug fix:change port value

* change chart script:Submit a stable version script

* fix bug:Implement option interface

* fix bug:change K8sDR.Register

* change config.yaml

* change chats script:minio service add ingress

* change chats script:minio service add ingress

* change chats script:kafka.replicaCount=3& change minio.api ingress

* delete change chats script

* change config.yaml

* change openim.yaml

* merge go.sum

* Add monitoring function and struct for Prometheus on gin and GRPC

* Add GRPC and gin server monitoring logic

* Add GRPC and gin server monitoring logic2

* Add GRPC and gin server monitoring logic3

* Add GRPC and gin server monitoring logic4

* Add GRPC and gin server monitoring logic5

* Add GRPC and gin server monitoring logic6

* Add GRPC and gin server monitoring logic7

* delete:old monitoring code

* add for test

* fix bug:change packname

* fix bug:delete getPromPort funciton

* fix bug:delete getPromPort funciton

* fix bug:change logs

* fix bug:change registerName logic in GetGrpcCusMetrics function

* add getPrometheus url api

* fix:config path logic

* fix:prometheus enable function

* fix:prometheus enable function

* fix:transfer Multi process monitoring logic

* del:del not using manifest

* fix:openim-msgtransfer.sh

* fix:openim-msgtransfer.sh

* cicd: robot automated Change

* delete not using files

* add prometheus docker-compose for monitor

* fix prometheus.yaml

* fix environment.sh

* fix init-config.sh

* fix init-config.sh

* fix env_template.yaml

* fix docker-compose.yml

* fix docker-compose.yml

* add openim_admin_front service

* change openim-admin-front

* del not using files

* add node-exporter-dashaboard.yaml

* fix: fix prom config file them

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>

* fix: fix prom config file them

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>

* feat: Complete the source code module design of monitoring

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>

* feat: Complete the source code module docs

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>

* feat: Complete the source code module docs

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>

* feat: add openim prometheus

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>

---------

Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com>
Co-authored-by: lin.huang <lin.huang@apulis.com>
Co-authored-by: xuexihuang <1339326187@qq.com>
Co-authored-by: xuexihuang <xuexihuang@users.noreply.github.com>
This commit is contained in:
Xinwei Xiong
2023-11-15 10:38:21 +08:00
committed by GitHub
parent 7502b4ac0f
commit ae048417ee
22 changed files with 2014 additions and 8765 deletions
+12 -3
View File
@@ -13,8 +13,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# 本脚本功能:根据 scripts/environment.sh 配置,生成 OPENIM 组件 YAML 配置文件。
# 示例./scripts/genconfig.sh scripts/install/environment.sh scripts/template/config.yaml
# Function of this script: Generate the OPENIM component YAML configuration file according to the scripts/environment.sh configuration.
# eg./scripts/genconfig.sh scripts/install/environment.sh scripts/template/config.yaml
# Read: https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/init-config.md
env_file="$1"
@@ -29,8 +29,17 @@ if [ $# -ne 2 ];then
exit 1
fi
# Check if the required commands exist
openim::util::require-dig
result=$?
if [ $result -ne 0 ]; then
openim::log::info "Please install 'dig' to use this feature."
openim::log::info "Installation instructions:"
openim::log::info " For Ubuntu/Debian: sudo apt-get install dnsutils"
openim::log::info " For CentOS/RedHat: sudo yum install bind-utils"
openim::log::info " For macOS: 'dig' should be preinstalled. If missing, try: brew install bind"
openim::log::info " For Windows: Install BIND9 tools from https://www.isc.org/download/"
openim::log::error_exit "Error: 'dig' command is required but not installed."
fi
source "${env_file}"
+2
View File
@@ -31,6 +31,7 @@ readonly ENV_FILE=${ENV_FILE:-"${OPENIM_ROOT}/scripts/install/environment.sh"}
declare -A TEMPLATES=(
["${OPENIM_ROOT}/deployments/templates/env_template.yaml"]="${OPENIM_ROOT}/.env"
["${OPENIM_ROOT}/deployments/templates/openim.yaml"]="${OPENIM_ROOT}/config/config.yaml"
["${OPENIM_ROOT}/deployments/templates/prometheus.yml"]="${OPENIM_ROOT}/config/prometheus.yml"
)
for template in "${!TEMPLATES[@]}"; do
@@ -45,6 +46,7 @@ for template in "${!TEMPLATES[@]}"; do
openim::log::error "Error processing template file ${template}"
exit 1
}
sleep 0.5
done
done
+14 -3
View File
@@ -116,7 +116,10 @@ LAST_OCTET=$((LAST_OCTET + 1))
PROMETHEUS_NETWORK_ADDRESS=$(generate_ip)
LAST_OCTET=$((LAST_OCTET + 1))
GRAFANA_NETWORK_ADDRESS=$(generate_ip)
LAST_OCTET=$((LAST_OCTET + 1))
NODE_EXPORTER_NETWORK_ADDRESS=$(generate_ip)
LAST_OCTET=$((LAST_OCTET + 1))
OPENIM_ADMIN_FRONT_NETWORK_ADDRESS=$(generate_ip)
###################### openim 配置 ######################
# read: https://github.com/openimsdk/open-im-server/blob/main/deployment/README.md
def "OPENIM_DATA_DIR" "/data/openim"
@@ -242,6 +245,9 @@ def "OPENIM_WEB_PORT" "11001" # openim-web的端口
def "OPENIM_WEB_ADDRESS" "${DOCKER_BRIDGE_GATEWAY}" # openim-web的地址
def "OPENIM_WEB_DIST_PATH" "/app/dist" # openim-web的dist路径
###################### openim-admin-front 配置信息 ######################
def "OPENIM_ADMIN_FRONT_PORT" "11002" # openim-admin-front的端口
###################### RPC 配置信息 ######################
def "RPC_REGISTER_IP" # RPC的注册IP
def "RPC_LISTEN_IP" "0.0.0.0" # RPC的监听IP
@@ -250,6 +256,9 @@ def "RPC_LISTEN_IP" "0.0.0.0" # RPC的监听IP
def "PROMETHEUS_PORT" "19090" # Prometheus的端口
def "PROMETHEUS_ADDRESS" "${DOCKER_BRIDGE_GATEWAY}" # Prometheus的地址
###################### node-exporter 配置 ######################
def "NODE_EXPORTER_PORT" "19100" # node-exporter的端口
def "NODE_EXPORTER_ADDRESS" "${DOCKER_BRIDGE_GATEWAY}" # node-exporter的地址
###################### Grafana 配置信息 ######################
def "GRAFANA_PORT" "3000" # Grafana的端口
def "GRAFANA_ADDRESS" "${DOCKER_BRIDGE_GATEWAY}" # Grafana的地址
@@ -340,8 +349,9 @@ def "IOS_BADGE_COUNT" "true" # IOS徽章计数
def "IOS_PRODUCTION" "false" # IOS生产
###################### Prometheus 配置信息 ######################
def "PROMETHEUS_ENABLE" "false" # 是否启用 Prometheus
def "PROMETHEUS_URL" "/prometheus"
# 是否启用 Prometheus
readonly PROMETHEUS_ENABLE=${PROMETHEUS_ENABLE:-'false'}
def "PROMETHEUS_URL" "${GRAFANA_ADDRESS}:${GRAFANA_PORT}"
# Api 服务的 Prometheus 端口
readonly API_PROM_PORT=${API_PROM_PORT:-'20100'}
# User 服务的 Prometheus 端口
@@ -367,6 +377,7 @@ readonly THIRD_PROM_PORT=${THIRD_PROM_PORT:-'21301'}
# Message Transfer 服务的 Prometheus 端口列表
readonly MSG_TRANSFER_PROM_PORT=${MSG_TRANSFER_PROM_PORT:-'21400, 21401, 21402, 21403'}
readonly MSG_TRANSFER_PROM_ADDRESS_PORT=${MSG_TRANSFER_PROM_ADDRESS_PORT:-"${DOCKER_BRIDGE_GATEWAY}:21400, ${DOCKER_BRIDGE_GATEWAY}:21401, ${DOCKER_BRIDGE_GATEWAY}:21402, ${DOCKER_BRIDGE_GATEWAY}:21403"}
###################### OpenIM openim-api ######################
def "OPENIM_API_HOST" "127.0.0.1"
+2 -2
View File
@@ -76,8 +76,8 @@ function openim::install::install_openim() {
openim::log::info "check openim dependency"
openim::common::sudo "cp -r ${OPENIM_ROOT}/config/* ${OPENIM_CONFIG_DIR}/"
echo ${LINUX_PASSWORD} | sudo -S bash -c \
"${OPENIM_ROOT}/scripts/genconfig.sh ${ENV_FILE} deployments/templates/openim.yaml > ${OPENIM_CONFIG_DIR}/config.yaml"
${OPENIM_ROOT}/scripts/genconfig.sh ${ENV_FILE} ${OPENIM_ROOT}/deployments/templates/openim.yaml > ${OPENIM_CONFIG_DIR}/config.yaml
${OPENIM_ROOT}/scripts/genconfig.sh ${ENV_FILE} ${OPENIM_ROOT}/deployments/templates/prometheus.yml > ${OPENIM_CONFIG_DIR}/prometheus.yml
openim::util::check_ports ${OPENIM_DEPENDENCY_PORT_LISTARIES[@]}
+1
View File
@@ -123,6 +123,7 @@ function openim::msggateway::status() {
# Check the running status of the ${SERVER_NAME}. If active (running) is displayed, the ${SERVER_NAME} is started successfully.
systemctl status ${SERVER_NAME}|grep -q 'active' || {
openim::log::error "${SERVER_NAME} failed to start, maybe not installed properly"
return 1
}
+3 -9
View File
@@ -1121,8 +1121,7 @@ function openim::util::check-file-in-alphabetical-order {
# Checks whether jq is installed.
function openim::util::require-jq {
if ! command -v jq &>/dev/null; then
echo "jq not found. Please install." 1>&2
return 1
openim::log::errexit "jq not found. Please install." 1>&2
fi
}
@@ -1130,15 +1129,10 @@ function openim::util::require-jq {
# Checks whether dig is installed and provides installation instructions if it is not.
function openim::util::require-dig {
if ! command -v dig &>/dev/null; then
echo "dig command not found."
echo "Please install 'dig' to use this feature."
echo "Installation instructions:"
echo " For Ubuntu/Debian: sudo apt-get install dnsutils"
echo " For CentOS/RedHat: sudo yum install bind-utils"
echo " For macOS: 'dig' should be preinstalled. If missing, try: brew install bind"
echo " For Windows: Install BIND9 tools from https://www.isc.org/download/"
openim::log::error "dig command not found."
return 1
fi
return 0
}
# outputs md5 hash of $1, works on macOS and Linux