feat: Enhancements to OpenIM Engineering Practices with Standardizer and Tool Versioning (#2159)

* feat: add standardizer optimize makefile

* feat: add standardizer optimize makefile

* feat: add openim test docs

* feat: add openim kafka docs

* feat: add openim kafka docs

* feat: add openim kafka docs
This commit is contained in:
Xinwei Xiong
2024-04-02 11:31:25 +08:00
committed by GitHub
parent 48df76fb8b
commit 4f40022105
28 changed files with 472 additions and 355 deletions
+162
View File
@@ -0,0 +1,162 @@
# OpenIM Kafka Guide
This document aims to provide a set of concise guidelines to help you quickly install and use Kafka through Docker Compose.
## Installing Kafka
With the Docker Compose script provided by OpenIM, you can easily install Kafka. Use the following command to start Kafka:
```bash
docker compose up -d
```
After executing this command, Kafka will be installed and started. You can confirm the Kafka container is running with the following command:
```bash
docker ps | grep kafka
```
The output of this command, as shown below, displays the status information of the Kafka container:
```
be416b5a0851 bitnami/kafka:3.5.1 "/opt/bitnami/script…" 3 days ago Up 2 days 9092/tcp, 0.0.0.0:19094->9094/tcp, :::19094->9094/tcp kafka
```
### References
- Official Docker installation documentation: [Click here](http://events.jianshu.io/p/b60afa35303a)
- Detailed installation guide: [Tutorial on Towards Data Science](https://towardsdatascience.com/how-to-install-apache-kafka-using-docker-the-easy-way-4ceb00817d8b)
## Using Kafka
### Entering the Kafka Container
To execute Kafka commands, you first need to enter the Kafka container. Use the following command:
```bash
docker exec -it kafka bash
```
### Kafka Command Tools
Inside the Kafka container, you can use various command-line tools to manage Kafka. These tools include but are not limited to:
- `kafka-topics.sh`: For creating, deleting, listing, or altering topics.
- `kafka-console-producer.sh`: Allows sending messages to a specified topic from the command line.
- `kafka-console-consumer.sh`: Allows reading messages from the command line, with the ability to specify topics.
- `kafka-consumer-groups.sh`: For managing consumer group information.
### Kafka Client Tool Installation
For easier Kafka management, you can install Kafka client tools. If you installed Kafka through OpenIM's Docker Compose, you can install the Kafka client tools with the following command:
```bash
make install.kafkactl
```
### Automatic Topic Creation
When installing Kafka through OpenIM's Docker Compose method, OpenIM automatically creates the following topics:
- `latestMsgToRedis`
- `msgToPush`
- `offlineMsgToMongoMysql`
These topics are created using the `scripts/create-topic.sh` script. The script waits for Kafka to be ready before executing the commands to create topics:
```bash
# Wait for Kafka to be ready
until /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092; do
echo "Waiting for Kafka to be ready..."
sleep 2
done
# Create topics
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic latestMsgToRedis
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic msgToPush
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic offlineMsgToMongoMysql
echo "Topics created."
```
The optimized and expanded documentation further details some basic commands for operations inside the Kafka container, as well as basic commands for managing Kafka using `kafkactl`. Here is a more detailed guide.
## Basic Commands in the Kafka Container
### Listing Topics
To list all existing topics, you can use the following command:
```bash
kafka-topics.sh --list --bootstrap-server localhost:9092
```
### Creating a New Topic
When creating a new topic, you can specify the number of partitions and the replication factor. Here is the command to create a new topic:
```bash
kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic your_topic_name
```
### Producing Messages
To send messages to a specific topic, you can use the producer command. The following command prompts you to enter messages, which are sent to the specified topic with each press of the Enter key:
```bash
kafka-console-producer.sh --broker-list localhost:9092 --topic your_topic_name
```
### Consuming Messages
To read messages from a specific topic, you can use the consumer command. The following command reads new messages from the specified topic and outputs them on the console:
```bash
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic your_topic_name --from-beginning
```
The `
--from-beginning` parameter reads messages from the beginning of the topic. If this parameter is omitted, only new messages will be read.
## Basic Commands Using `kafkactl`
`kafkactl` is a command-line tool for managing and operating Kafka clusters. It offers a more modern way to interact with Kafka.
### Listing Topics
To list all topics, you can use:
```bash
kafkactl get topics
```
### Creating a New Topic
To create a new topic with `kafkactl`, use:
```bash
kafkactl create topic your_topic_name --partitions 1 --replication-factor 1
```
### Producing Messages
To send messages to a topic, you can use:
```bash
kafkactl produce your_topic_name --value "your message"
```
Here, `"your message"` is the content of the message you want to send.
### Consuming Messages
To consume messages from a topic, use:
```bash
kafkactl consume your_topic_name --from-beginning
```
Again, the `--from-beginning` parameter will start consuming messages from the beginning of the topic. If you do not wish to start from the beginning, you can omit this parameter.
+1 -1
View File
@@ -2,7 +2,7 @@
## Script Logging Documentation Link
If you wish to view the script's logging documentation, you can click on this link: [Logging Documentation](https://github.com/OpenIMSDK/Open-IM-Server/blob/main/docs/contrib/bash-log.md).
If you wish to view the script's logging documentation, you can click on this link: [Logging Documentation](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/bash-log.md).
Below is the documentation for logging and error handling in the OpenIM Go project.
+83 -3
View File
@@ -2,14 +2,94 @@
This document serves as a comprehensive guide to understanding and utilizing the `test.sh` script for testing OpenIM RPC services. The `test.sh` script is a collection of bash functions designed to test various aspects of the OpenIM RPC services, ensuring that each part of the API is functioning as expected.
+ Scriptshttps://github.com/OpenIMSDK/Open-IM-Server/tree/main/scripts/install/test.sh
+ Scriptshttps://github.com/openimsdk/open-im-server/tree/main/scripts/install/test.sh
For some complex, bulky functional tests, performance tests, and various e2e tests, We are all in the current warehouse to https://github.com/OpenIMSDK/Open-IM-Server/tree/main/test or https://github.com/openim-sigs/test-infra directory In the.
For some complex, bulky functional tests, performance tests, and various e2e tests, We are all in the current warehouse to https://github.com/openimsdk/open-im-server/tree/main/test or https://github.com/openim-sigs/test-infra directory In the.
+ About OpenIM Feature [Test Docs](https://docs.google.com/spreadsheets/d/1zELWkwxgOOZ7u5pmYCqqaFnvZy2SVajv/edit?usp=sharing&ouid=103266350914914783293&rtpof=true&sd=true)
## Util Test
## Usage
Let's restructure and enhance the document under a unified second-level heading, adding clarity and details for better comprehension and visual appeal.
---
## Development Guide
### Comprehensive Testing Instructions
#### Running Unit Tests
- **Command**: To execute unit tests, input the following in your terminal:
```
make test
```
#### Evaluating Test Coverage
- **Overview**: It's crucial to assess how much of your code is covered by tests.
- **Command**:
```bash
make cover
```
This command generates a report detailing the percentage of your code tested, ensuring adherence to quality standards.
#### Conducting API Tests
- **Purpose**: API tests validate the interaction and functionality of your application's interfaces.
- **How to Run**:
```
make test-api
```
Use this to check the integrity and reliability of your API endpoints.
#### End-to-End (E2E) Testing
- **Scope**: E2E tests simulate real-user scenarios from start to finish.
- **Execution**:
```
make test-e2e
```
This comprehensive testing ensures your application performs as expected in real-world situations.
### Crafting Unit Test Cases
#### Setup for Test Case Generation
- **Installation**: Install the `gotests` tool to generate test cases automatically.
```bash
make install.gotests
```
This command installs the `gotests` tool for test case generation.
- **Environment Preparation**: Define your test template environment variable and generate test cases as shown below:
```bash
export GOTESTS_TEMPLATE=testify
gotests -i -w -only keyFunc .
```
This prepares your environment for test case generation using the `testify` template.
#### Isolating Function Tests
- **Single Function Testing**: When you need to focus on testing a single function for detailed examination.
- **Method**:
```bash
go test -v -run TestKeyFunc
```
This command specifically runs tests for `TestKeyFunc`, allowing targeted debugging and validation.
### Important Note
- **Quality Assurance**: Throughout your development process, it is imperative to ensure that the unit test coverage meets or surpasses the standards set by OpenIM.
- **Maintaining Standards**: Regularly running your tests with
```make test```
supports maintaining high code quality and adherence to OpenIM's rigorous testing benchmarks.
## E2E Test
TODO
## Api Test
The `test.sh` script is located within the `./scripts/install/` directory of the OpenIM service's codebase. To use the script, navigate to this directory from your terminal:
Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

Before

Width:  |  Height:  |  Size: 207 KiB

After

Width:  |  Height:  |  Size: 207 KiB

Before

Width:  |  Height:  |  Size: 118 KiB

After

Width:  |  Height:  |  Size: 118 KiB