mirror of
https://github.com/openimsdk/open-im-server.git
synced 2026-04-28 14:29:19 +08:00
🚀 Enhancements in Offline Deployment Documentation, CICD Process Upgrades, Deployment Optimization, and Additional Fixes (#1721)
* feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: support openim prom enbels Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add go formity checker Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim offline deployment docs Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim offline deployment docs Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim offline deployment docs Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim support arm arch Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * fix: fox openim support arm arch Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * fix: fox openim support arm arch Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim v3.4 and v3.5 changelog Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add devcontainer.json file Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim v3.4 and v3.5 changelog Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: add openim v3.4 and v3.5 changelog Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * Update util.sh amend shell get pid command fd on mac. * feat: Add To start developing OpenIM Docs Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * docs: update openim readme and dov container Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: set openim lint Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com> * feat: fix go release scripts Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: fix go release scripts Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * docs(changelog): add CHANGELOG-3.5.0+6.a09200e.md * docs(changelog): add CHANGELOG-3.5.0+2.e0bd54f.md * docs(changelog): add CHANGELOG-3.5.0+2.e0bd54f-1-g4ce6a0fa6.md * feat: set openim lint Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com> * docs(changelog): add CHANGELOG-3.5.0+2.e0bd54f-3-g52f9fc209.md * docs(changelog): add CHANGELOG-3.5.0+2.e0bd54f-4-g87f685b17.md * docs(changelog): add CHANGELOG-3.5.0.md * feat: set openim lint Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com> * fix: fix scripts functions upload Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * fix: fix scripts functions upload Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * docs(changelog): add CHANGELOG-3.5.0.md * fix: fix scripts functions upload Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * fix: fix scripts functions upload Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> * feat: set openim lint Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com> --------- Signed-off-by: Xinwei Xiong (cubxxw) <3293172751nss@gmail.com> Signed-off-by: Xinwei Xiong(cubxxw) <3293172751nss@gmail.com> Co-authored-by: mrwenhui <mr.wenhui@gmail.com>
This commit is contained in:
Xinwei Xiong
@@ -108,7 +108,7 @@ export CHAT_IMAGE_VERSION="main"
|
||||
export SERVER_IMAGE_VERSION="main"
|
||||
```
|
||||
|
||||
These variables are stored within the [`environment.sh`](https://github.com/OpenIMSDK/openim-docker/blob/main/scripts/install/environment.sh) configuration:
|
||||
These variables are stored within the [`environment.sh`](https://github.com/OpenIMSDK/open-im-server/blob/main/scripts/install/environment.sh) configuration:
|
||||
|
||||
```bash
|
||||
readonly CHAT_IMAGE_VERSION=${CHAT_IMAGE_VERSION:-'main'}
|
||||
@@ -448,7 +448,7 @@ This section involves configuring the log settings, including storage location,
|
||||
|
||||
| Parameter | Example Value | Description |
|
||||
| ------------------------- | ------------------------ | --------------------------------- |
|
||||
| LOG_STORAGE_LOCATION | ""${OPENIM_ROOT}"/logs/" | Location for storing logs |
|
||||
| LOG_STORAGE_LOCATION | "${OPENIM_ROOT}/logs/" | Location for storing logs |
|
||||
| LOG_ROTATION_TIME | "24" | Log rotation time (in hours) |
|
||||
| LOG_REMAIN_ROTATION_COUNT | "2" | Number of log rotations to retain |
|
||||
| LOG_REMAIN_LOG_LEVEL | "6" | Log level to retain |
|
||||
|
||||
@@ -75,7 +75,7 @@ It is critical that our full community is actively engaged on enhancements in th
|
||||
|
||||
- Be aware the cherry pick script assumes you have a git remote called `upstream` that points at the openim-server github org.
|
||||
|
||||
Please see our [recommended Git workflow](https://github.com/openimsdk/open-im-server/blob/main/docs/contributors/github-workflow.md#workflow).
|
||||
Please see our [recommended Git workflow](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/github-workflow.md#workflow).
|
||||
|
||||
- You will need to run the cherry pick script separately for each patch release you want to cherry pick to. Cherry picks should be applied to all [active](https://github.com/openimsdk/open-im-server/releases) release branches where the fix is applicable.
|
||||
|
||||
|
||||
@@ -8,7 +8,6 @@ description: |
|
||||
commit hygiene.
|
||||
---
|
||||
|
||||
|
||||
|
||||
## 1. Fork in the cloud
|
||||
|
||||
@@ -28,10 +27,10 @@ neither `$GOPATH/src/github.com/${your github profile name}/`
|
||||
nor any other pattern will work.
|
||||
|
||||
```sh
|
||||
export working_dir="$(go env GOPATH)/src/k8s.io"
|
||||
export working_dir="$(go env GOPATH)/src/github.com/openimsdk"
|
||||
```
|
||||
|
||||
If you already do Go development on github, the `k8s.io` directory
|
||||
If you already do Go development on github, the `github.com/openimsdk` directory
|
||||
will be a sibling to your existing `github.com` directory.
|
||||
|
||||
Set `user` to match your github profile name:
|
||||
|
||||
@@ -23,8 +23,8 @@ In the `scripts/init_config.sh` file, we defined some template files. These temp
|
||||
```
|
||||
# Defines an associative array where the keys are the template files and the values are the corresponding output files.
|
||||
declare -A TEMPLATES=(
|
||||
[""${OPENIM_ROOT}"/scripts/template/config-tmpl/env.template"]="${OPENIM_OUTPUT_SUBPATH}/bin/.env"
|
||||
[""${OPENIM_ROOT}"/scripts/template/config-tmpl/config.yaml"]="${OPENIM_OUTPUT_SUBPATH}/bin/config.yaml"
|
||||
["${OPENIM_ROOT}/scripts/template/config-tmpl/env.template"]="${OPENIM_OUTPUT_SUBPATH}/bin/.env"
|
||||
["${OPENIM_ROOT}/scripts/template/config-tmpl/config.yaml"]="${OPENIM_OUTPUT_SUBPATH}/bin/config.yaml"
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
# Mac Developer Deployment Guide for OpenIM
|
||||
|
||||
## Introduction
|
||||
|
||||
This guide aims to assist Mac-based developers in contributing effectively to OpenIM. It covers the setup of a development environment tailored for Mac, including the use of GitHub for development workflow and `devcontainer` for a consistent development experience.
|
||||
|
||||
Before contributing to OpenIM through issues and pull requests, make sure you are familiar with GitHub and the [pull request workflow](https://docs.github.com/en/get-started/quickstart/github-flow).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### System Requirements
|
||||
|
||||
- macOS (latest stable version recommended)
|
||||
- Internet connection
|
||||
- Administrator access
|
||||
|
||||
### Knowledge Requirements
|
||||
|
||||
- Basic understanding of Git and GitHub
|
||||
- Familiarity with Docker and containerization
|
||||
- Experience with Go programming language
|
||||
|
||||
## Setting up the Development Environment
|
||||
|
||||
### Installing Homebrew
|
||||
|
||||
Homebrew is an essential package manager for macOS. Install it using:
|
||||
|
||||
```sh
|
||||
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
|
||||
```
|
||||
|
||||
### Installing and Configuring Git
|
||||
|
||||
1. Install Git:
|
||||
|
||||
```sh
|
||||
brew install git
|
||||
```
|
||||
|
||||
2. Configure Git with your user details:
|
||||
|
||||
```sh
|
||||
git config --global user.name "Your Name"
|
||||
git config --global user.email "your.email@example.com"
|
||||
```
|
||||
|
||||
### Forking and Cloning the Repository
|
||||
|
||||
1. Fork the OpenIM repository on GitHub to your account.
|
||||
2. Clone your fork to your local machine:
|
||||
|
||||
```sh
|
||||
git clone https://github.com/<your-username>/open-im-server.git
|
||||
# The Docker bridging network mode for Mac is slightly different and needs to be set:
|
||||
export DOCKER_BRIDGE_SUBNET=127.0.0.0/16
|
||||
# Set OpenIM IP
|
||||
export OPENIM_IP=<your-ip>
|
||||
# Init Config
|
||||
make init
|
||||
|
||||
# Start Components
|
||||
docker compose up -d
|
||||
|
||||
# Start OpenIM Server
|
||||
make start
|
||||
```
|
||||
|
||||
### Setting Up the Devcontainer
|
||||
|
||||
`Devcontainers` provide a Docker-based isolated development environment.
|
||||
|
||||
Read [README.md](https://github.com/openimsdk/open-im-server/tree/main/.devcontainer) in the `.devcontainer` directory of the project to learn more about the devcontainer.
|
||||
|
||||
To set it up:
|
||||
|
||||
1. Install Docker Desktop for Mac from [Docker Hub](https://docs.docker.com/desktop/install/mac-install/).
|
||||
2. Install Visual Studio Code and the Remote - Containers extension.
|
||||
3. Open the cloned OpenIM repository in VS Code.
|
||||
4. VS Code will prompt to reopen the project in a container. Accept this to set up the environment automatically.
|
||||
|
||||
### Installing Go and Dependencies
|
||||
|
||||
Use Homebrew to install Go:
|
||||
|
||||
```sh
|
||||
brew install go
|
||||
```
|
||||
|
||||
Ensure the version of Go is compatible with the version required by OpenIM (refer to the main documentation for version requirements).
|
||||
|
||||
### Additional Tools
|
||||
|
||||
Install other required tools like Docker, Vagrant, and necessary GNU utils as described in the main documentation.
|
||||
|
||||
## GitHub Development Workflow
|
||||
|
||||
### Creating a New Branch
|
||||
|
||||
For new features or fixes, create a new branch:
|
||||
|
||||
```sh
|
||||
git checkout -b feat/your-feature-name
|
||||
```
|
||||
|
||||
### Making Changes and Committing
|
||||
|
||||
1. Make your changes in the code.
|
||||
2. Stage your changes:
|
||||
|
||||
```sh
|
||||
git add .
|
||||
```
|
||||
|
||||
3. Commit with a meaningful message:
|
||||
|
||||
```sh
|
||||
git commit -m "Add a brief description of your changes"
|
||||
```
|
||||
|
||||
### Pushing Changes and Creating Pull Requests
|
||||
|
||||
1. Push your branch to GitHub:
|
||||
|
||||
```sh
|
||||
git push origin feat/your-feature-name
|
||||
```
|
||||
|
||||
2. Go to your fork on GitHub and create a pull request to the main OpenIM repository.
|
||||
|
||||
### Keeping Your Fork Updated
|
||||
|
||||
Regularly sync your fork with the main repository:
|
||||
|
||||
```sh
|
||||
git fetch upstream
|
||||
git checkout main
|
||||
git rebase upstream/main
|
||||
```
|
||||
|
||||
More read: [https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md](https://github.com/openimsdk/open-im-server/blob/main/CONTRIBUTING.md)
|
||||
|
||||
## Testing and Quality Assurance
|
||||
|
||||
Run tests as described in the OpenIM documentation to ensure your changes do not break existing functionality.
|
||||
|
||||
## Conclusion
|
||||
|
||||
This guide provides a comprehensive overview for Mac developers to set up and contribute to OpenIM. By following these steps, you can ensure a smooth and efficient development experience. Happy coding!
|
||||
@@ -6,25 +6,57 @@
|
||||
|
||||
Below are the base images and their versions you'll need:
|
||||
|
||||
- wurstmeister/kafka
|
||||
- redis:7.0.0
|
||||
- mongo:6.0.2
|
||||
- mysql:5.7
|
||||
- wurstmeister/zookeeper
|
||||
- minio/minio
|
||||
- [ ] bitnami/kafka:3.5.1
|
||||
- [ ] redis:7.0.0
|
||||
- [ ] mongo:6.0.2
|
||||
- [ ] bitnami/zookeeper:3.8
|
||||
- [ ] minio/minio:latest
|
||||
|
||||
> [!IMPORTANT]
|
||||
> It is important to note that OpenIM removed mysql components from versions v3.5.0 (release-v3.5) and above, so mysql can be deployed without this requirement or above
|
||||
|
||||
**If you need to install more IM components or monitoring products：**
|
||||
|
||||
OpenIM:
|
||||
|
||||
> [!TIP]
|
||||
> If you need to install more IM components or monitoring products [images.md](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/images.md)
|
||||
|
||||
- [ ] ghcr.io/openimsdk/openim-web:latest
|
||||
- [ ] ghcr.io/openimsdk/openim-admin:latest
|
||||
- [ ] ghcr.io/openimsdk/openim-chat:latest
|
||||
- [ ] ghcr.io/openimsdk/openim-server:latest
|
||||
|
||||
|
||||
Monitoring:
|
||||
|
||||
- [ ] prom/prometheus：v2.48.1
|
||||
- [ ] prom/alertmanager：v0.23.0
|
||||
- [ ] grafana/grafana：10.2.2
|
||||
- [ ] bitnami/node-exporter：1.7.0
|
||||
|
||||
|
||||
Use the following commands to pull these base images:
|
||||
|
||||
```
|
||||
docker pull wurstmeister/kafka
|
||||
```bash
|
||||
docker pull bitnami/kafka:3.5.1
|
||||
docker pull redis:7.0.0
|
||||
docker pull mongo:6.0.2
|
||||
docker pull mysql:5.7
|
||||
docker pull wurstmeister/zookeeper
|
||||
docker pull minio/minio
|
||||
docker pull bitnami/zookeeper:3.8
|
||||
docker pull minio/minio:latest
|
||||
```
|
||||
|
||||
## 2. OpenIM & Chat Images
|
||||
If you need to install more IM components or monitoring products:
|
||||
|
||||
```bash
|
||||
docker pull prom/prometheus:v2.48.1
|
||||
docker pull prom/alertmanager:v0.23.0
|
||||
docker pull grafana/grafana:10.2.2
|
||||
docker pull bitnami/node-exporter:1.7.0
|
||||
```
|
||||
|
||||
## 2. OpenIM Images
|
||||
|
||||
**For detailed understanding of version management and storage of OpenIM and Chat**: [version.md](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/version.md)
|
||||
|
||||
@@ -42,9 +74,26 @@ docker pull ghcr.io/openimsdk/openim-server:<version-name>
|
||||
- Execute the following command to pull the image:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/openimsdk/openim-server:<version-name>
|
||||
docker pull ghcr.io/openimsdk/openim-chat:<version-name>
|
||||
```
|
||||
|
||||
### Web Image
|
||||
|
||||
- Execute the following command to pull the image:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/openimsdk/openim-web:<version-name>
|
||||
```
|
||||
|
||||
### Admin Image
|
||||
|
||||
- Execute the following command to pull the image:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/openimsdk/openim-admin:<version-name>
|
||||
```
|
||||
|
||||
|
||||
## 3. Image Storage Selection
|
||||
|
||||
**Repositories**:
|
||||
@@ -71,46 +120,61 @@ You can select from the following versions:
|
||||
1. **Pull images**: Execute the above `docker pull` commands to pull all required images locally.
|
||||
2. **Save images**:
|
||||
|
||||
```
|
||||
```bash
|
||||
docker save -o <tar-file-name>.tar <image-name>
|
||||
```
|
||||
|
||||
1. **Fetch code**: Clone the repository:
|
||||
If you want to save all the images, use the following command:
|
||||
|
||||
```
|
||||
git clone https://github.com/OpenIMSDK/openim-docker.git
|
||||
```bash
|
||||
docker save -o <tar-file-name>.tar $(docker images -q)
|
||||
```
|
||||
|
||||
Or download the code from [Releases](https://github.com/OpenIMSDK/openim-docker/releases/).
|
||||
|
||||
1. **Transfer files**: Use `scp` to transfer all images and code to the intranet server.
|
||||
3. **Fetch code**: Clone the repository:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/openimsdk/openim-docker.git
|
||||
```
|
||||
|
||||
Or download the code from [Releases](https://github.com/openimsdk/openim-docker/releases/).
|
||||
|
||||
> Because of the difference between win and linux newlines, please do not clone the repository with win and then synchronize scp to linux.
|
||||
|
||||
4. **Transfer files**: Use `scp` to transfer all images and code to the intranet server.
|
||||
|
||||
```bash
|
||||
scp <tar-file-name>.tar user@remote-ip:/path/on/remote/server
|
||||
```
|
||||
|
||||
Or choose other transfer methods such as a hard drive.
|
||||
|
||||
1. **Import images**: On the intranet server:
|
||||
5. **Import images**: On the intranet server:
|
||||
|
||||
```
|
||||
```bash
|
||||
docker load -i <tar-file-name>.tar
|
||||
```
|
||||
|
||||
1. **Deploy**: Navigate to the `openim-docker` repository directory and follow the README guide for deployment.
|
||||
2. **Deploy using Docker-compose**:
|
||||
Import directly with shortcut commands:
|
||||
|
||||
```bash
|
||||
for i in `ls ./`;do docker load -i $i;done
|
||||
```
|
||||
docker-compose up -d
|
||||
|
||||
# Verify
|
||||
docker-compose ps
|
||||
6. **Deploy**: Navigate to the `openim-docker` repository directory and follow the [README guide](https://github.com/openimsdk/openim-docker) for deployment.
|
||||
|
||||
7. **Deploy using docker compose**:
|
||||
|
||||
```bash
|
||||
export OPENIM_IP="your ip" # Set Ip
|
||||
make init # Init config
|
||||
docker compose up -d # Deployment
|
||||
docker compose ps # Verify
|
||||
```
|
||||
|
||||
> **Note**: If you're using a version of Docker prior to 20, make sure you've installed `docker-compose`.
|
||||
|
||||
## 6. Reference Links
|
||||
|
||||
- [OpenIMSDK Issue #432](https://github.com/openimsdk/open-im-server/issues/432)
|
||||
- [openimsdk Issue #432](https://github.com/openimsdk/open-im-server/issues/432)
|
||||
- [Notion Link](https://nsddd.notion.site/435ee747c0bc44048da9300a2d745ad3?pvs=25)
|
||||
- [OpenIMSDK Issue #474](https://github.com/openimsdk/open-im-server/issues/474)
|
||||
- [openimsdk Issue #474](https://github.com/openimsdk/open-im-server/issues/474)
|
||||
@@ -111,32 +111,35 @@ Importing Grafana Dashboards is a straightforward process and is applicable to O
|
||||
|
||||
To monitor OpenIM in Grafana, you need to focus on three categories of key metrics, each with its specific deployment and configuration steps:
|
||||
|
||||
1. **OpenIM Metrics (`prometheus-dashboard.yaml`)**:
|
||||
+ **Configuration File Path**: Located at `config/prometheus-dashboard.yaml`.
|
||||
+ **Enabling Monitoring**: Set the environment variable `export PROMETHEUS_ENABLE=true` to enable Prometheus monitoring.
|
||||
+ **More Information**: Refer to the [OpenIM Configuration Guide](https://docs.openim.io/configurations/prometheus-integration).
|
||||
2. **Node Exporter**:
|
||||
+ **Container Deployment**: Deploy the `quay.io/prometheus/node-exporter` container for node monitoring.
|
||||
+ **Get Dashboard**: Access the [Node Exporter Full Feature Dashboard](https://grafana.com/grafana/dashboards/1860-node-exporter-full/) and import it using YAML file download or ID import.
|
||||
+ **Deployment Guide**: Refer to the [Node Exporter Deployment Documentation](https://prometheus.io/docs/guides/node-exporter/).
|
||||
3. **Middleware Metrics**: Each middleware requires specific steps and configurations to enable monitoring. Here is a list of common middleware and links to their respective setup guides:
|
||||
+ MySQL:
|
||||
+ **Configuration**: Ensure MySQL has performance monitoring enabled.
|
||||
+ **Link**: Refer to the [MySQL Monitoring Configuration Guide](https://grafana.com/docs/grafana/latest/datasources/mysql/).
|
||||
+ Redis:
|
||||
+ **Configuration**: Configure Redis to allow monitoring data export.
|
||||
+ **Link**: Refer to the [Redis Monitoring Guide](https://grafana.com/docs/grafana/latest/datasources/redis/).
|
||||
+ MongoDB:
|
||||
+ **Configuration**: Set up monitoring metrics for MongoDB.
|
||||
+ **Link**: Refer to the [MongoDB Monitoring Guide](https://grafana.com/grafana/plugins/grafana-mongodb-datasource/).
|
||||
+ Kafka:
|
||||
+ **Configuration**: Integrate Kafka with Prometheus monitoring.
|
||||
+ **Link**: Refer to the [Kafka Monitoring Guide](https://grafana.com/grafana/plugins/grafana-kafka-datasource/).
|
||||
+ Zookeeper:
|
||||
+ **Configuration**: Ensure Zookeeper can be monitored by Prometheus.
|
||||
+ **Link**: Refer to the [Zookeeper Monitoring Configuration](https://grafana.com/docs/grafana/latest/datasources/zookeeper/).
|
||||
**OpenIM Metrics (`prometheus-dashboard.yaml`)**:
|
||||
|
||||
- **Configuration File Path**: Find this at `config/prometheus-dashboard.yaml`.
|
||||
- **Enabling Monitoring**: Activate Prometheus monitoring by setting the environment variable: `export PROMETHEUS_ENABLE=true`.
|
||||
- **More Information**: For detailed instructions, see the [OpenIM Configuration Guide](https://docs.openim.io/configurations/prometheus-integration).
|
||||
|
||||
**Node Exporter**:
|
||||
|
||||
- **Container Deployment**: Use the container `quay.io/prometheus/node-exporter` for effective node monitoring.
|
||||
- **Access Dashboard**: Visit the [Node Exporter Full Feature Dashboard](https://grafana.com/grafana/dashboards/1860-node-exporter-full/) for dashboard integration either through YAML file download or ID.
|
||||
- **Deployment Guide**: For deployment steps, consult the [Node Exporter Deployment Documentation](https://prometheus.io/docs/guides/node-exporter/).
|
||||
|
||||
**Middleware Metrics**: Different middlewares require unique steps and configurations for monitoring:
|
||||
|
||||
- MySQL:
|
||||
- **Configuration**: Make sure MySQL is set up for performance monitoring.
|
||||
- **Guide**: See the [MySQL Monitoring Configuration Guide](https://grafana.com/docs/grafana/latest/datasources/mysql/).
|
||||
- Redis:
|
||||
- **Configuration**: Adjust Redis settings to enable monitoring data export.
|
||||
- **Guide**: Consult the [Redis Monitoring Guide](https://grafana.com/docs/grafana/latest/datasources/redis/).
|
||||
- MongoDB:
|
||||
- **Configuration**: Configure MongoDB for monitoring metrics.
|
||||
- **Guide**: Visit the [MongoDB Monitoring Guide](https://grafana.com/grafana/plugins/grafana-mongodb-datasource/).
|
||||
- Kafka:
|
||||
- **Configuration**: Set up Kafka for Prometheus monitoring integration.
|
||||
- **Guide**: Refer to the [Kafka Monitoring Guide](https://grafana.com/grafana/plugins/grafana-kafka-datasource/).
|
||||
- Zookeeper:
|
||||
- **Configuration**: Ensure Prometheus can monitor Zookeeper.
|
||||
- **Guide**: Check out the [Zookeeper Monitoring Configuration](https://grafana.com/docs/grafana/latest/datasources/zookeeper/).
|
||||
|
||||
**Importing Steps**:
|
||||
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
# OpenIM Release Automation Design Document
|
||||
|
||||
This document outlines the automation process for releasing OpenIM. You can use the `make release` command for automated publishing. We will discuss how to use the `make release` command and Github Actions CICD separately, while also providing insight into the design principles involved.
|
||||
|
||||
## Github Actions Automation
|
||||
|
||||
In our CICD pipeline, we have implemented logic for automating the release process using the goreleaser tool. To achieve this, follow these steps on your local machine or server:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/openimsdk/open-im-server
|
||||
cd open-im-server
|
||||
git tag -a v3.6.0 -s -m "release: xxx"
|
||||
# For pre-release versions: git tag -a v3.6.0-rc.0 -s -m "pre-release: xxx"
|
||||
git push origin v3.6.0
|
||||
```
|
||||
|
||||
The remaining tasks are handled by automated processes:
|
||||
|
||||
+ Automatically complete the release publication on Github
|
||||
+ Automatically build the `v3.6.0` version image and push it to aliyun, dockerhub, and github
|
||||
|
||||
Through these automated steps, we achieve rapid and efficient OpenIM version releases, simplifying the release process and enhancing productivity.
|
||||
|
||||
|
||||
Certainly, here is the continuation of the document in English:
|
||||
|
||||
## Local Make Release Design
|
||||
|
||||
There are two primary scenarios for local usage:
|
||||
|
||||
+ Advanced compilation and release, manually executed locally
|
||||
+ Quick compilation verification and version release, manually executed locally
|
||||
|
||||
**These two scenarios can also be combined, for example, by tagging locally and then releasing:**
|
||||
|
||||
```bash
|
||||
git add .
|
||||
git commit -a -s -m "release(v3.6.0): ......"
|
||||
git tag v3.6.0
|
||||
git release
|
||||
git push origin main
|
||||
```
|
||||
|
||||
In a local environment, you can use the `make release` command to complete the release process. The main implementation logic can be found in the `/data/workspaces/open-im-server/scripts/lib/release.sh` file. First, let's explore its usage through the help information.
|
||||
|
||||
### Help Information
|
||||
|
||||
To view the help information, execute the following command:
|
||||
|
||||
```bash
|
||||
$ ./scripts/release.sh --help
|
||||
Usage: release.sh [options]
|
||||
Options:
|
||||
-h, --help Display this help message
|
||||
-se, --setup-env Execute environment setup
|
||||
-vp, --verify-prereqs Execute prerequisite verification
|
||||
-bc, --build-command Execute build command
|
||||
-bi, --build-image Execute build image (default is not executed)
|
||||
-pt, --package-tarballs Execute tarball packaging
|
||||
-ut, --upload-tarballs Execute tarball upload
|
||||
-gr, --github-release Execute GitHub release
|
||||
-gc, --generate-changelog Execute changelog generation
|
||||
```
|
||||
|
||||
### Default Behavior
|
||||
|
||||
If no options are provided, all operations are executed by default:
|
||||
|
||||
```bash
|
||||
# If no options are provided, enable all operations by default
|
||||
if [ "$#" -eq 0 ]; then
|
||||
perform_setup_env=true
|
||||
perform_verify_prereqs=true
|
||||
perform_build_command=true
|
||||
perform_package_tarballs=true
|
||||
perform_upload_tarballs=true
|
||||
perform_github_release=true
|
||||
perform_generate_changelog=true
|
||||
# TODO: Defaultly not enable build_image
|
||||
# perform_build_image=true
|
||||
fi
|
||||
```
|
||||
|
||||
### Environment Variable Setup
|
||||
|
||||
Before starting, you need to set environment variables:
|
||||
|
||||
```bash
|
||||
export TENCENT_SECRET_KEY=OZZ****************************
|
||||
export TENCENT_SECRET_ID=AKI****************************
|
||||
```
|
||||
|
||||
### Modifying COS Account and Password
|
||||
|
||||
If you need to change the COS account, password, and bucket information, please modify the following section in the `/data/workspaces/open-im-server/scripts/lib/release.sh` file:
|
||||
|
||||
```bash
|
||||
readonly BUCKET="openim-1306374445"
|
||||
readonly REGION="ap-guangzhou"
|
||||
readonly COS_RELEASE_DIR="openim-release"
|
||||
```
|
||||
|
||||
### GitHub Release Configuration
|
||||
|
||||
If you intend to use the GitHub Release feature, you also need to set the environment variable:
|
||||
|
||||
```bash
|
||||
export GITHUB_TOKEN="your_github_token"
|
||||
```
|
||||
|
||||
### Modifying GitHub Release Basic Information
|
||||
|
||||
If you need to modify the basic information of GitHub Release, please edit the following section in the `/data/workspaces/open-im-server/scripts/lib/release.sh` file:
|
||||
|
||||
```bash
|
||||
# OpenIM GitHub account information
|
||||
readonly OPENIM_GITHUB_ORG=openimsdk
|
||||
readonly OPENIM_GITHUB_REPO=open-im-server
|
||||
```
|
||||
|
||||
This setup allows you to configure and execute the local release process according to your specific needs.
|
||||
|
||||
|
||||
### GitHub Release Versioning Rules
|
||||
|
||||
Firstly, it's important to note that GitHub Releases should primarily be for pre-release versions. However, goreleaser might provide a `prerelease: auto` option, which automatically marks versions with pre-release indicators like `-rc1`, `-beta`, etc., as pre-releases.
|
||||
|
||||
So, if your most recent tag does not have pre-release indicators such as `-rc1` or `-beta`, even if you use `make release` for pre-release versions, goreleaser might still consider them as formal releases.
|
||||
|
||||
To avoid this issue, I have added the `--draft` flag to github-release. This way, all releases are created as drafts.
|
||||
|
||||
## CICD Release Documentation Design
|
||||
|
||||
The release records still require manual composition for GitHub Release. This is different from github-release.
|
||||
|
||||
This approach ensures that all releases are initially created as drafts, allowing you to manually review and edit the release documentation on GitHub. This manual step provides more control and allows you to curate release notes and other information before making them public.
|
||||
|
||||
|
||||
## Makefile Section
|
||||
|
||||
This document aims to elaborate and explain key sections of the OpenIM Release automation design, including the Makefile section and functions within the code. Below, we will provide a detailed explanation of the logic and functions of each section.
|
||||
|
||||
In the project's root directory, the Makefile imports a subdirectory:
|
||||
|
||||
```makefile
|
||||
include scripts/make-rules/release.mk
|
||||
```
|
||||
|
||||
And defines the `release` target as follows:
|
||||
|
||||
```makefile
|
||||
## release: release the project ✨
|
||||
.PHONY: release release: release.verify release.ensure-tag
|
||||
@scripts/release.sh
|
||||
```
|
||||
|
||||
### Importing Subdirectory
|
||||
|
||||
At the beginning of the Makefile, the `include scripts/make-rules/release.mk` statement imports the `release.mk` file from the subdirectory. This file contains rules and configurations related to releases to be used in subsequent operations.
|
||||
|
||||
### The `release` Target
|
||||
|
||||
The Makefile defines a target named `release`, which is used to execute the project's release operation. This target is marked as a phony target (`.PHONY`), meaning it doesn't represent an actual file or directory but serves as an identifier for executing a series of actions.
|
||||
|
||||
In the `release` target, two dependency targets are executed first: `release.verify` and `release.ensure-tag`. Afterward, the `scripts/release.sh` script is called to perform the actual release operation.
|
||||
|
||||
## Logic of `release.verify` and `release.ensure-tag`
|
||||
|
||||
```makefile
|
||||
## release.verify: Check if a tool is installed and install it
|
||||
.PHONY: release.verify
|
||||
release.verify: tools.verify.git-chglog tools.verify.github-release tools.verify.coscmd tools.verify.coscli
|
||||
|
||||
## release.ensure-tag: ensure tag
|
||||
.PHONY: release.ensure-tag
|
||||
release.ensure-tag: tools.verify.gsemver
|
||||
@scripts/ensure-tag.sh
|
||||
```
|
||||
|
||||
### `release.verify` Target
|
||||
|
||||
The `release.verify` target is used to check and install tools. It depends on four sub-targets: `tools.verify.git-chglog`, `tools.verify.github-release`, `tools.verify.coscmd`, and `tools.verify.coscli`. These sub-targets aim to check if specific tools are installed and attempt to install them if they are not.
|
||||
|
||||
The purpose of this target is to ensure that the necessary tools required for the release process are available so that subsequent operations can be executed smoothly.
|
||||
|
||||
### `release.ensure-tag` Target
|
||||
|
||||
The `release.ensure-tag` target is used to ensure that the project has a version tag. It depends on the sub-target `tools.verify.gsemver`, indicating that it should check if the `gsemver` tool is installed before executing.
|
||||
|
||||
When the `release.ensure-tag` target is executed, it calls the `scripts/ensure-tag.sh` script to ensure that the project has a version tag. Version tags are typically used to identify specific versions of the project for management and release in version control systems.
|
||||
|
||||
## Logic of `release.sh` Script
|
||||
|
||||
```bash
|
||||
openim::golang::setup_env
|
||||
openim::build::verify_prereqs
|
||||
openim::release::verify_prereqs
|
||||
#openim::build::build_image
|
||||
openim::build::build_command
|
||||
openim::release::package_tarballs
|
||||
openim::release::upload_tarballs
|
||||
git push origin ${VERSION}
|
||||
#openim::release::github_release
|
||||
#openim::release::generate_changelog
|
||||
```
|
||||
|
||||
The `release.sh` script is responsible for executing the actual release operations. Below is the logic of this script:
|
||||
|
||||
1. `openim::golang::setup_env`: This function sets up some configurations for the Golang development environment.
|
||||
|
||||
2. `openim::build::verify_prereqs`: This function is used to verify whether the prerequisites for building are met. This includes checking dependencies, environment variables, and more.
|
||||
|
||||
3. `openim::release::verify_prereqs`: Similar to the previous function, this one is used to verify whether the prerequisites for the release are met. It focuses on conditions relevant to the release.
|
||||
|
||||
4. `openim::build::build_command`: This function is responsible for building the project's command, which typically involves compiling the project or performing other build operations.
|
||||
|
||||
5. `openim::release::package_tarballs`: This function is used to package tarball files required for the release. These tarballs are usually used for distribution packages during the release.
|
||||
|
||||
6. `openim::release::upload_tarballs`: This function is used to upload the packaged tarball files, typically to a distribution platform or repository.
|
||||
|
||||
7. `git push origin ${VERSION}`: This line of command pushes the version tag to the remote Git repository's `origin` branch, marking this release in the version control system.
|
||||
|
||||
In the comments, you can see that there are some operations that are commented out, such as `openim::build::build_image`, `openim::release::github_release`, and `openim::release::generate_changelog`. These operations are related to building images, releasing to GitHub, and generating changelogs, and they can be enabled in the release process as needed.
|
||||
|
||||
Let's take a closer look at the function responsible for packaging the tarball files:
|
||||
|
||||
```bash
|
||||
function openim::release::package_tarballs() {
|
||||
# Clean out any old releases
|
||||
rm -rf "${RELEASE_STAGE}" "${RELEASE_TARS}" "${RELEASE_IMAGES}"
|
||||
mkdir -p "${RELEASE_TARS}"
|
||||
openim::release::package_src_tarball &
|
||||
openim::release::package_client_tarballs &
|
||||
openim::release::package_openim_manifests_tarball &
|
||||
openim::release::package_server_tarballs &
|
||||
openim::util::wait-for-jobs || { openim::log::error "previous tarball phase failed"; return 1; }
|
||||
|
||||
openim::release::package_final_tarball & # _final depends on some of the previous phases
|
||||
openim::util::wait-for-jobs || { openim::log::error "previous tarball phase failed"; return 1; }
|
||||
}
|
||||
```
|
||||
|
||||
The `openim::release::package_tarballs()` function is responsible for packaging the tarball files required for the release. Here is the specific logic of this function:
|
||||
|
||||
1. `rm -rf "${RELEASE_STAGE}" "${RELEASE_TARS}" "${RELEASE_IMAGES}"`: First, the function removes any old release directories and files to ensure a clean starting state.
|
||||
|
||||
2. `mkdir -p "${RELEASE_TARS}"`: Next, it creates a directory `${RELEASE_TARS}` to store the packaged tarball files. If the directory doesn't exist, it will be created.
|
||||
|
||||
3. `openim::release::package_final_tarball &`: This is an asynchronous operation that depends on some of the previous phases. It is likely used to package the final tarball file, which includes the contents of all previous asynchronous operations.
|
||||
|
||||
4. `openim::util::wait-for-jobs`: It waits for all asynchronous operations to complete. If any of the previous asynchronous operations fail, an error will be returned.
|
||||
Reference in New Issue
Block a user