✨Large refactoring projects: OpenIM automation, scripting, and openimctl refactoring (#825)

* fix: fix bin tools path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: fix golang release file path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: fix golang release file path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: fix scripts and optimize Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: fix scripts path module Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: sync script code Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add lib and start scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * ci: add copyright scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * ci: add go-docs file and copyright scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add scripts cross ower Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * style: Formatting code make lint path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * style: Formatting code make lint path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * style: Formatting code make lint path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * style: Formatting code make lint path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: chat scripts path bug Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: channge smail images Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add makefile feature Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add config and images log Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * style: Migrate directory to remove docker to images Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * style: formatting style Code Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: set opneim's bash logs Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: option scripts and docs Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add git cherry Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add git cherry Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save all bash and docs labels Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: scripts feature extend Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add config path config Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add config path config Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add feat scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add save scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add save scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add sctips help Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add start sctips help Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save scripts file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save all file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add openim server template file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add alot of system design Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save all file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save all file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add env config options Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add more robot details Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add more module explain Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add scripts environment details design Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add openim msgtransfer scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add openim msgtransfer scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add more design scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add file save Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add file save Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add rpc build and start Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add rpc build and start Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add rpc build and start Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save all images file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add scripts set Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add test options Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: fix config path file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: update config file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: update config file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: update config file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: update config file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add readme docs Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save build scripts Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add all actions file Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add chat scripts name Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add all compose Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: commit tag Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save server code Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add docker compose fix Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add docker compose fix Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add docker compose fix Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: save server code Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: optimize dockerfile option Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: optimize dockerfile option Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: optimize dockerfile option Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add all options Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add all options Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add more scrips Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add more options Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add more options Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add config path Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: Add some optimizations Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * fix: Add some optimizations Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * Delete go.work.sum * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * Update .env * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add docker compose fix Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: add docker compose fix Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> * feat: delele go work sum Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com> --------- Signed-off-by: Xinwei Xiong(cubxxw-openim) <3293172751nss@gmail.com>
2026-05-08 02:55:58 +08:00 · 2023-08-23 09:09:51 +08:00
parent ff9b258229
commit 58f591e5f6
255 changed files with 12457 additions and 4111 deletions
@@ -0,0 +1,47 @@
+## OpenIM Logging System: Design and Usage
+
+**PATH:** `scripts/lib/logging.sh`
+
+### Introduction
+
+OpenIM, an intricate project, requires a robust logging mechanism to diagnose issues, maintain system health, and provide insights. A custom-built logging system embedded within OpenIM ensures consistent and structured logs. Let's delve into the design of this logging system and understand its various functions and their usage scenarios.
+
+### Design Overview
+
+1. **Initialization**: The system begins by determining the verbosity level through the `OPENIM_VERBOSE` variable. If it's not set, a default value of 5 is assigned. This verbosity level dictates the depth of the log details.
+2. **Log File Setup**: Logs are stored in the directory specified by `OPENIM_OUTPUT`. If this variable isn't explicitly set, it defaults to the `_output` directory relative to the script location. Each log file is named based on the date to facilitate easy identification.
+3. **Logging Function**: The `echo_log()` function plays a pivotal role by writing messages to both the console (stdout) and the log file.
+4. **Logging to a file**: The `echo_log()` function writes to the log file by appending the message to the file. It also adds a timestamp to the message. path: `_output/logs/*`, Enable logging by default. Set to false to disable. If you wish to turn off output to log files set `ENABLE_LOGGING=flase`. 
+
+### Key Functions & Their Usages
+
+1. **Error Handling**:
+   - `openim::log::errexit()`: Activated when a command exits with an error. It prints a call tree showing the sequence of functions leading to the error and then calls `openim::log::error_exit()` with relevant information.
+   - `openim::log::install_errexit()`: Sets up the trap for catching errors and ensures that the error handler (`errexit`) gets propagated to various script constructs like functions, expansions, and subshells.
+2. **Logging Levels**:
+   - `openim::log::error()`: Logs error messages with a timestamp. The log message starts with '!!!' to indicate its severity.
+   - `openim::log::info()`: Provides informational messages. The display of these messages is governed by the verbosity level (`OPENIM_VERBOSE`).
+   - `openim::log::progress()`: Designed for logging progress messages or creating progress bars.
+   - `openim::log::status()`: Logs status messages with a timestamp, prefixing each entry with '+++' for easy identification.
+   - `openim::log::success()`: Highlights successful operations with a bright green prefix. It's ideal for visually signifying operations that completed successfully.
+3. **Exit and Stack Trace**:
+   - `openim::log::error_exit()`: Logs an error message, dumps the call stack, and exits the script with a specified exit code.
+   - `openim::log::stack()`: Prints out a stack trace, showing the call hierarchy leading to the point where this function was invoked.
+4. **Usage Information**:
+   - `openim::log::usage() & openim::log::usage_from_stdin()`: Both functions provide a mechanism to display usage instructions. The former accepts arguments directly, while the latter reads them from stdin.
+5. **Test Function**:
+   - `openim::log::test_log()`: This function is a test suite to verify that all logging functions are operating as expected.
+
+### Usage Scenario
+
+Imagine a situation where an OpenIM operation fails, and you need to ascertain the cause. With the logging system in place, you can:
+
+- Check the log file for the specific day to find error messages with the '!!!' prefix.
+- View the call tree and stack trace to trace back the sequence of operations leading to the failure.
+- Use the verbosity level to filter out unnecessary details and focus on the crux of the issue.
+
+This systematic and structured approach greatly simplifies the debugging process, making system maintenance more efficient.
+
+### Conclusion
+
+OpenIM's logging system is a testament to the importance of structured and detailed logging in complex projects. By using this logging mechanism, developers and system administrators can streamline troubleshooting and ensure the seamless operation of the OpenIM project.
@@ -0,0 +1,129 @@
+# Continuous Integration and Automation
+
+Every change on the OpenIM repository, either made through a pull request or direct push, triggers the continuous integration pipelines defined within the same repository. Needless to say, all the OpenIM contributions can be merged until all the checks pass (AKA having green builds).
+
+- [Continuous Integration and Automation](#continuous-integration-and-automation)
+  - [CI Platforms](#ci-platforms)
+    - [GitHub Actions](#github-actions)
+  - [Running locally](#running-locally)
+
+## CI Platforms
+
+Currently, there are two different platforms involved in running the CI processes:
+
+- GitHub actions
+- Drone pipelines on CNCF infrastructure
+
+### GitHub Actions
+
+All the existing GitHub Actions are defined as YAML files under the `.github/workflows` directory. These can be grouped into:
+
+- **PR Checks**. These actions run all the required validations upon PR creation and update. Covering the DCO compliance check, `x86_64` test batteries (unit, integration, smoke), and code coverage.
+- **Repository automation**. Currently, it only covers issues and epic grooming.
+
+Everything runs on GitHub's provided runners; thus, the tests are limited to run in `x86_64` architectures.
+
+
+## Running locally
+
+A contributor should verify their changes locally to speed up the pull request process. Fortunately, all the CI steps can be on local environments, except for the publishing ones, through either of the following methods:
+
+**User Makefile:**
+```bash
+root@PS2023EVRHNCXG:~/workspaces/openim/Open-IM-Server# make help 😊
+
+Usage: make <TARGETS> <OPTIONS> ...
+
+Targets:
+
+all                          Run tidy, gen, add-copyright, format, lint, cover, build 🚀
+build                        Build binaries by default 🛠️
+multiarch                    Build binaries for multiple platforms. See option PLATFORMS. 🌍
+tidy                         tidy go.mod ✨
+vendor                       vendor go.mod 📦
+style                        code style -> fmt,vet,lint 💅
+fmt                          Run go fmt against code. ✨
+vet                          Run go vet against code. ✅
+lint                         Check syntax and styling of go sources. ✔️
+format                       Gofmt (reformat) package sources (exclude vendor dir if existed). 🔄
+test                         Run unit test. 🧪
+cover                        Run unit test and get test coverage. 📊
+updates                      Check for updates to go.mod dependencies 🆕
+imports                      task to automatically handle import packages in Go files using goimports tool 📥
+clean                        Remove all files that are created by building. 🗑️
+image                        Build docker images for host arch. 🐳
+image.multiarch              Build docker images for multiple platforms. See option PLATFORMS. 🌍🐳
+push                         Build docker images for host arch and push images to registry. 📤🐳
+push.multiarch               Build docker images for multiple platforms and push images to registry. 🌍📤🐳
+tools                        Install dependent tools. 🧰
+gen                          Generate all necessary files. 🧩
+swagger                      Generate swagger document. 📖
+serve-swagger                Serve swagger spec and docs. 🚀📚
+verify-copyright             Verify the license headers for all files. ✅
+add-copyright                Add copyright ensure source code files have license headers. 📄
+release                      release the project 🎉
+help                         Show this help info. ℹ️
+help-all                     Show all help details info. ℹ️📚
+
+Options:
+
+DEBUG            Whether or not to generate debug symbols. Default is 0. ❓
+
+BINS             Binaries to build. Default is all binaries under cmd. 🛠️
+This option is available when using: make {build}(.multiarch) 🧰
+Example: make build BINS="openim-api openim_cms_api".
+
+PLATFORMS        Platform to build for. Default is linux_arm64 and linux_amd64. 🌍
+This option is available when using: make {build}.multiarch 🌍
+Example: make multiarch PLATFORMS="linux_s390x linux_mips64
+linux_mips64le darwin_amd64 windows_amd64 linux_amd64 linux_arm64".
+
+V                Set to 1 enable verbose build. Default is 0. 📝
+```
+
+
+How to Use Makefile to Help Contributors Build Projects Quickly 😊
+
+The `make help` command is a handy tool that provides useful information on how to utilize the Makefile effectively. By running this command, contributors will gain insights into various targets and options available for building projects swiftly.
+
+Here's a breakdown of the targets and options provided by the Makefile:
+
+**Targets 😃**
+
+1. `all`: This target runs multiple tasks like `tidy`, `gen`, `add-copyright`, `format`, `lint`, `cover`, and `build`. It ensures comprehensive project building.
+2. `build`: The primary target that compiles binaries by default. It is particularly useful for creating the necessary executable files.
+3. `multiarch`: A target that builds binaries for multiple platforms. Contributors can specify the desired platforms using the `PLATFORMS` option.
+4. `tidy`: This target cleans up the `go.mod` file, ensuring its consistency.
+5. `vendor`: A target that updates the project dependencies based on the `go.mod` file.
+6. `style`: Checks the code style using tools like `fmt`, `vet`, and `lint`. It ensures a consistent coding style throughout the project.
+7. `fmt`: Formats the code using the `go fmt` command, ensuring proper indentation and formatting.
+8. `vet`: Runs the `go vet` command to identify common errors in the code.
+9. `lint`: Validates the syntax and styling of Go source files using a linter.
+10. `format`: Reformats the package sources using `gofmt`. It excludes the vendor directory if it exists.
+11. `test`: Executes unit tests to ensure the functionality and stability of the code.
+12. `cover`: Performs unit tests and calculates the test coverage of the code.
+13. `updates`: Checks for updates to the project's dependencies specified in the `go.mod` file.
+14. `imports`: Automatically handles import packages in Go files using the `goimports` tool.
+15. `clean`: Removes all files generated during the build process, effectively cleaning up the project directory.
+16. `image`: Builds Docker images for the host architecture.
+17. `image.multiarch`: Similar to the `image` target, but it builds Docker images for multiple platforms. Contributors can specify the desired platforms using the `PLATFORMS` option.
+18. `push`: Builds Docker images for the host architecture and pushes them to a registry.
+19. `push.multiarch`: Builds Docker images for multiple platforms and pushes them to a registry. Contributors can specify the desired platforms using the `PLATFORMS` option.
+20. `tools`: Installs the necessary tools or dependencies required by the project.
+21. `gen`: Generates all the required files automatically.
+22. `swagger`: Generates the swagger document for the project.
+23. `serve-swagger`: Serves the swagger specification and documentation.
+24. `verify-copyright`: Verifies the license headers for all project files.
+25. `add-copyright`: Adds copyright headers to the source code files.
+26. `release`: Releases the project, presumably for distribution.
+27. `help`: Displays information about available targets and options.
+28. `help-all`: Shows detailed information about all available targets and options.
+
+**Options 😄**
+
+1. `DEBUG`: A boolean option that determines whether or not to generate debug symbols. The default value is 0 (false).
+2. `BINS`: Specifies the binaries to build. By default, it builds all binaries under the `cmd` directory. Contributors can provide a list of specific binaries using this option.
+3. `PLATFORMS`: Specifies the platforms to build for. The default platforms are `linux_arm64` and `linux_amd64`. Contributors can specify multiple platforms by providing a space-separated list of platform names.
+4. `V`: A boolean option that enables verbose build output when set to 1 (true). The default value is 0 (false).
+
+With these targets and options in place, contributors can efficiently build projects using the Makefile. Happy coding! 🚀😊
@@ -39,9 +39,6 @@ import (
 _ "github.com/jinzhu/gorm/dialects/mysql"

 // inner package
-v1 "github.com/marmotedu/api/apiserver/v1"
-metav1 "github.com/marmotedu/apimachinery/pkg/meta/v1"
-"github.com/marmotedu/iam/pkg/cli/genericclioptions"
 )
 ```