Compare commits

...

3 Commits

Author SHA1 Message Date
Gordon 9673f50518 feat: send message add sendTime field for import messages. (#1104)
* fix: to start im or chat, ZooKeeper must be started first.

* fix: msg gateway start output err info

Signed-off-by: Gordon <1432970085@qq.com>

* fix: msg gateway start output err info

Signed-off-by: Gordon <1432970085@qq.com>

* chore: package path changes

Signed-off-by: withchao <993506633@qq.com>

* fix: go mod update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* chore: package path changes

Signed-off-by: withchao <993506633@qq.com>

* chore: package path changes

Signed-off-by: withchao <993506633@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: get all userID

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: msggateway add online status call

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: log change

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: log change

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* chore: network mode change

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* feat: add api of get server time

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* feat: remove go work sum

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: pull message add isRead field

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: check msg-transfer script

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: start don't kill old process

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: check component

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: pull message set isRead only message come from single.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: add ex field to update group info.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

* cicd: robot automated Change

* refactor: change project module name.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: change project module name.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: change project module name.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

* test: for pressure test.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* test: for pressure test.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* test: for pressure test.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* test: message log.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

* fxi: component check output valid info.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fxi: component check output valid info.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* test: send message test log.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

* cicd: robot automated Change

* test: remove info log.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* feat: api of send message add sendTime field.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

---------

Signed-off-by: Gordon <1432970085@qq.com>
Signed-off-by: withchao <993506633@qq.com>
Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: withchao <993506633@qq.com>
Co-authored-by: Xinwei Xiong <3293172751NSS@gmail.com>
Co-authored-by: FGadvancer <FGadvancer@users.noreply.github.com>
2023-09-20 04:04:25 +00:00
withchao 5c31d12253 feat: s3 public read (#1080)
* fix: repeated modification session notification

* fix: repeated modification session notification

* fix: jpush return a nil pointer panic

* fix: push redis pkg

* fix: OANotification

* feat: add rpc GetConversationNeedOfflinePushUserIDs

* update pkg

* cicd: robot automated Change

* offlinePushMsg

* conversation

* conversation

* cicd: robot automated Change

* conversation

* cicd: robot automated Change

* conversation

* url 2 im s3

* url 2 im s3

* cicd: robot automated Change

* url 2 im s3

* s3 public read

* cicd: robot automated Change

* s3 public read

* cicd: robot automated Change

* s3 public read

* s3 public read

* s3 public read

* s3 public read

* s3 public read

* cicd: robot automated Change

* s3 public read

* s3 public read

* fix: SendMsg api

* config scripts

* config scripts

---------

Co-authored-by: withchao <withchao@users.noreply.github.com>
2023-09-19 15:23:46 +08:00
Gordon bba662f404 Feature: add log support for developer (#1101)
* fix: to start im or chat, ZooKeeper must be started first.

* fix: msg gateway start output err info

Signed-off-by: Gordon <1432970085@qq.com>

* fix: msg gateway start output err info

Signed-off-by: Gordon <1432970085@qq.com>

* chore: package path changes

Signed-off-by: withchao <993506633@qq.com>

* fix: go mod update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* chore: package path changes

Signed-off-by: withchao <993506633@qq.com>

* chore: package path changes

Signed-off-by: withchao <993506633@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: token update

Signed-off-by: Gordon <1432970085@qq.com>

* fix: get all userID

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: msggateway add online status call

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: log change

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: log change

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* chore: network mode change

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* feat: add api of get server time

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* feat: remove go work sum

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: pull message add isRead field

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: check msg-transfer script

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix:  script update

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: start don't kill old process

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: check component

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: pull message set isRead only message come from single.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: multiple gateway kick user each other.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: add ex field to update group info.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

* cicd: robot automated Change

* refactor: change project module name.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: change project module name.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* refactor: change project module name.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* cicd: robot automated Change

* test: for pressure test.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* test: for pressure test.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* test: for pressure test.

Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>

* fix: to start im or chat, ZooKeeper must be started first.

---------

Signed-off-by: Gordon <1432970085@qq.com>
Signed-off-by: withchao <993506633@qq.com>
Signed-off-by: Gordon <46924906+FGadvancer@users.noreply.github.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: withchao <993506633@qq.com>
Co-authored-by: Xinwei Xiong <3293172751NSS@gmail.com>
Co-authored-by: FGadvancer <FGadvancer@users.noreply.github.com>
2023-09-19 15:08:05 +08:00
30 changed files with 440 additions and 103 deletions
+4 -1
View File
@@ -132,7 +132,7 @@ api:
# minio.signEndpoint is minio public network address # minio.signEndpoint is minio public network address
object: object:
enable: "minio" enable: "minio"
apiURL: "http://http://127.0.0.1:10002" apiURL: "http://127.0.0.1:10002"
minio: minio:
bucket: "openim" bucket: "openim"
endpoint: "http://172.28.0.1:10005" endpoint: "http://172.28.0.1:10005"
@@ -140,11 +140,13 @@ object:
secretAccessKey: "openIM123" secretAccessKey: "openIM123"
sessionToken: '' sessionToken: ''
signEndpoint: "http://127.0.0.1:10005" signEndpoint: "http://127.0.0.1:10005"
publicRead: false
cos: cos:
bucketURL: https://temp-1252357374.cos.ap-chengdu.myqcloud.com bucketURL: https://temp-1252357374.cos.ap-chengdu.myqcloud.com
secretID: '' secretID: ''
secretKey: '' secretKey: ''
sessionToken: '' sessionToken: ''
publicRead: false
oss: oss:
endpoint: "https://oss-cn-chengdu.aliyuncs.com" endpoint: "https://oss-cn-chengdu.aliyuncs.com"
bucket: "demo-9999999" bucket: "demo-9999999"
@@ -152,6 +154,7 @@ object:
accessKeyID: '' accessKeyID: ''
accessKeySecret: '' accessKeySecret: ''
sessionToken: '' sessionToken: ''
publicRead: false
###################### RPC Port Configuration ###################### ###################### RPC Port Configuration ######################
+3 -1
View File
@@ -140,11 +140,13 @@ object:
secretAccessKey: "${MINIO_SECRET_KEY}" secretAccessKey: "${MINIO_SECRET_KEY}"
sessionToken: ${MINIO_SESSION_TOKEN} sessionToken: ${MINIO_SESSION_TOKEN}
signEndpoint: "${MINIO_SIGN_ENDPOINT}" signEndpoint: "${MINIO_SIGN_ENDPOINT}"
publicRead: ${MINIO_PUBLIC_READ}
cos: cos:
bucketURL: ${COS_BUCKET_URL} bucketURL: ${COS_BUCKET_URL}
secretID: ${COS_SECRET_ID} secretID: ${COS_SECRET_ID}
secretKey: ${COS_SECRET_KEY} secretKey: ${COS_SECRET_KEY}
sessionToken: ${COS_SESSION_TOKEN} sessionToken: ${COS_SESSION_TOKEN}
publicRead: ${COS_PUBLIC_READ}
oss: oss:
endpoint: "${OSS_ENDPOINT}" endpoint: "${OSS_ENDPOINT}"
bucket: "${OSS_BUCKET}" bucket: "${OSS_BUCKET}"
@@ -152,7 +154,7 @@ object:
accessKeyID: ${OSS_ACCESS_KEY_ID} accessKeyID: ${OSS_ACCESS_KEY_ID}
accessKeySecret: ${OSS_ACCESS_KEY_SECRET} accessKeySecret: ${OSS_ACCESS_KEY_SECRET}
sessionToken: ${OSS_SESSION_TOKEN} sessionToken: ${OSS_SESSION_TOKEN}
publicRead: ${OSS_PUBLIC_READ}
###################### RPC Port Configuration ###################### ###################### RPC Port Configuration ######################
# RPC service ports # RPC service ports
+1 -1
View File
@@ -37,7 +37,7 @@ require github.com/google/uuid v1.3.1
require ( require (
github.com/IBM/sarama v1.41.1 github.com/IBM/sarama v1.41.1
github.com/OpenIMSDK/protocol v0.0.21 github.com/OpenIMSDK/protocol v0.0.23
github.com/OpenIMSDK/tools v0.0.14 github.com/OpenIMSDK/tools v0.0.14
github.com/aliyun/aliyun-oss-go-sdk v2.2.9+incompatible github.com/aliyun/aliyun-oss-go-sdk v2.2.9+incompatible
github.com/go-redis/redis v6.15.9+incompatible github.com/go-redis/redis v6.15.9+incompatible
+2 -2
View File
@@ -19,8 +19,8 @@ firebase.google.com/go v3.13.0+incompatible/go.mod h1:xlah6XbEyW6tbfSklcfe5FHJIw
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/IBM/sarama v1.41.1 h1:B4/TdHce/8Ipza+qrLIeNJ9D1AOxZVp/3uDv6H/dp2M= github.com/IBM/sarama v1.41.1 h1:B4/TdHce/8Ipza+qrLIeNJ9D1AOxZVp/3uDv6H/dp2M=
github.com/IBM/sarama v1.41.1/go.mod h1:JFCPURVskaipJdKRFkiE/OZqQHw7jqliaJmRwXCmSSw= github.com/IBM/sarama v1.41.1/go.mod h1:JFCPURVskaipJdKRFkiE/OZqQHw7jqliaJmRwXCmSSw=
github.com/OpenIMSDK/protocol v0.0.21 h1:5H6H+hJ9d/VgRqttvxD/zfK9Asd+4M8Eknk5swSbUVY= github.com/OpenIMSDK/protocol v0.0.23 h1:L545aRQez6Ro+AaJB1Z6Mz7ojnDtp41WqASxYveCkcE=
github.com/OpenIMSDK/protocol v0.0.21/go.mod h1:F25dFrwrIx3lkNoiuf6FkCfxuwf8L4Z8UIsdTHP/r0Y= github.com/OpenIMSDK/protocol v0.0.23/go.mod h1:F25dFrwrIx3lkNoiuf6FkCfxuwf8L4Z8UIsdTHP/r0Y=
github.com/OpenIMSDK/tools v0.0.14 h1:WLof/+WxyPyRST+QkoTKubYCiV73uCLiL8pgnpH/yKQ= github.com/OpenIMSDK/tools v0.0.14 h1:WLof/+WxyPyRST+QkoTKubYCiV73uCLiL8pgnpH/yKQ=
github.com/OpenIMSDK/tools v0.0.14/go.mod h1:eg+q4A34Qmu73xkY0mt37FHGMCMfC6CtmOnm0kFEGFI= github.com/OpenIMSDK/tools v0.0.14/go.mod h1:eg+q4A34Qmu73xkY0mt37FHGMCMfC6CtmOnm0kFEGFI=
github.com/QcloudApi/qcloud_sign_golang v0.0.0-20141224014652-e4130a326409/go.mod h1:1pk82RBxDY/JZnPQrtqHlUFfCctgdorsd9M06fMynOM= github.com/QcloudApi/qcloud_sign_golang v0.0.0-20141224014652-e4130a326409/go.mod h1:1pk82RBxDY/JZnPQrtqHlUFfCctgdorsd9M06fMynOM=
+3 -2
View File
@@ -58,7 +58,7 @@ func (m MessageApi) newUserSendMsgReq(c *gin.Context, params *apistruct.SendMsg)
options := make(map[string]bool, 5) options := make(map[string]bool, 5)
switch params.ContentType { switch params.ContentType {
case constant.Text: case constant.Text:
newContent = params.Content["text"].(string) fallthrough
case constant.Picture: case constant.Picture:
fallthrough fallthrough
case constant.Custom: case constant.Custom:
@@ -100,6 +100,7 @@ func (m MessageApi) newUserSendMsgReq(c *gin.Context, params *apistruct.SendMsg)
ContentType: params.ContentType, ContentType: params.ContentType,
Content: []byte(newContent), Content: []byte(newContent),
CreateTime: utils.GetCurrentTimestampByMill(), CreateTime: utils.GetCurrentTimestampByMill(),
SendTime: params.SendTime,
Options: options, Options: options,
OfflinePushInfo: params.OfflinePushInfo, OfflinePushInfo: params.OfflinePushInfo,
}, },
@@ -207,7 +208,6 @@ func (m *MessageApi) SendMessage(c *gin.Context) {
apiresp.GinError(c, errs.ErrArgs.WithDetail(err.Error()).Wrap()) apiresp.GinError(c, errs.ErrArgs.WithDetail(err.Error()).Wrap())
return return
} }
log.ZInfo(c, "SendMessage", "req", req)
if !authverify.IsAppManagerUid(c) { if !authverify.IsAppManagerUid(c) {
apiresp.GinError(c, errs.ErrNoPermission.Wrap("only app manager can send message")) apiresp.GinError(c, errs.ErrNoPermission.Wrap("only app manager can send message"))
return return
@@ -224,6 +224,7 @@ func (m *MessageApi) SendMessage(c *gin.Context) {
respPb, err := m.Client.SendMsg(c, sendMsgReq) respPb, err := m.Client.SendMsg(c, sendMsgReq)
if err != nil { if err != nil {
status = constant.MsgSendFailed status = constant.MsgSendFailed
log.ZError(c, "send message err", err)
apiresp.GinError(c, err) apiresp.GinError(c, err)
return return
} }
+5
View File
@@ -156,6 +156,11 @@ func NewGinRouter(discov discoveryregistry.SvcDiscoveryRegistry, rdb redis.Unive
thirdGroup.POST("/fcm_update_token", t.FcmUpdateToken) thirdGroup.POST("/fcm_update_token", t.FcmUpdateToken)
thirdGroup.POST("/set_app_badge", t.SetAppBadge) thirdGroup.POST("/set_app_badge", t.SetAppBadge)
logs := thirdGroup.Group("/logs")
logs.POST("/upload", t.UploadLogs)
logs.POST("/delete", t.DeleteLogs)
logs.POST("/search", t.SearchLogs)
objectGroup := r.Group("/object", ParseToken) objectGroup := r.Group("/object", ParseToken)
objectGroup.POST("/part_limit", t.PartLimit) objectGroup.POST("/part_limit", t.PartLimit)
+13
View File
@@ -105,3 +105,16 @@ func (o *ThirdApi) ObjectRedirect(c *gin.Context) {
} }
c.Redirect(http.StatusFound, resp.Url) c.Redirect(http.StatusFound, resp.Url)
} }
// #################### logs ####################
func (o *ThirdApi) UploadLogs(c *gin.Context) {
a2r.Call(third.ThirdClient.UploadLogs, o.Client, c)
}
func (o *ThirdApi) DeleteLogs(c *gin.Context) {
a2r.Call(third.ThirdClient.DeleteLogs, o.Client, c)
}
func (o *ThirdApi) SearchLogs(c *gin.Context) {
a2r.Call(third.ThirdClient.SearchLogs, o.Client, c)
}
+8 -3
View File
@@ -59,7 +59,7 @@ const (
PongMessage = 10 PongMessage = 10
) )
type PongHandler func(string) error type PingPongHandler func(string) error
type Client struct { type Client struct {
w *sync.Mutex w *sync.Mutex
@@ -107,8 +107,12 @@ func (c *Client) ResetClient(
c.token = token c.token = token
} }
func (c *Client) pongHandler(_ string) error { func (c *Client) pingHandler(_ string) error {
c.conn.SetReadDeadline(pongWait) c.conn.SetReadDeadline(pongWait)
err := c.writePongMsg()
if err != nil {
return err
}
return nil return nil
} }
@@ -122,10 +126,11 @@ func (c *Client) readMessage() {
}() }()
c.conn.SetReadLimit(maxMessageSize) c.conn.SetReadLimit(maxMessageSize)
_ = c.conn.SetReadDeadline(pongWait) _ = c.conn.SetReadDeadline(pongWait)
c.conn.SetPongHandler(c.pongHandler) c.conn.SetPingHandler(c.pingHandler)
for { for {
messageType, message, returnErr := c.conn.ReadMessage() messageType, message, returnErr := c.conn.ReadMessage()
if returnErr != nil { if returnErr != nil {
log.ZWarn(c.ctx, "readMessage", returnErr, "messageType", messageType)
c.closedErr = returnErr c.closedErr = returnErr
return return
} }
+6 -2
View File
@@ -41,7 +41,8 @@ type LongConn interface {
SetConnNil() SetConnNil()
// SetReadLimit sets the maximum size for a message read from the peer.bytes // SetReadLimit sets the maximum size for a message read from the peer.bytes
SetReadLimit(limit int64) SetReadLimit(limit int64)
SetPongHandler(handler PongHandler) SetPongHandler(handler PingPongHandler)
SetPingHandler(handler PingPongHandler)
// GenerateLongConn Check the connection of the current and when it was sent are the same // GenerateLongConn Check the connection of the current and when it was sent are the same
GenerateLongConn(w http.ResponseWriter, r *http.Request) error GenerateLongConn(w http.ResponseWriter, r *http.Request) error
} }
@@ -116,9 +117,12 @@ func (d *GWebSocket) SetReadLimit(limit int64) {
d.conn.SetReadLimit(limit) d.conn.SetReadLimit(limit)
} }
func (d *GWebSocket) SetPongHandler(handler PongHandler) { func (d *GWebSocket) SetPongHandler(handler PingPongHandler) {
d.conn.SetPongHandler(handler) d.conn.SetPongHandler(handler)
} }
func (d *GWebSocket) SetPingHandler(handler PingPongHandler) {
d.conn.SetPingHandler(handler)
}
//func (d *GWebSocket) CheckSendConnDiffNow() bool { //func (d *GWebSocket) CheckSendConnDiffNow() bool {
// return d.conn == d.sendConn // return d.conn == d.sendConn
+146
View File
@@ -0,0 +1,146 @@
package third
import (
"context"
"crypto/rand"
"fmt"
"time"
"github.com/OpenIMSDK/protocol/constant"
"github.com/OpenIMSDK/protocol/third"
"github.com/OpenIMSDK/tools/errs"
"github.com/OpenIMSDK/tools/utils"
utils2 "github.com/OpenIMSDK/tools/utils"
"github.com/openimsdk/open-im-server/v3/pkg/authverify"
relationtb "github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation"
)
func genLogID() string {
const dataLen = 10
data := make([]byte, dataLen)
rand.Read(data)
chars := []byte("0123456789")
for i := 0; i < len(data); i++ {
if i == 0 {
data[i] = chars[1:][data[i]%9]
} else {
data[i] = chars[data[i]%10]
}
}
return string(data)
}
func (t *thirdServer) UploadLogs(ctx context.Context, req *third.UploadLogsReq) (*third.UploadLogsResp, error) {
var DBlogs []*relationtb.Log
userID := ctx.Value(constant.OpUserID).(string)
platform := constant.PlatformID2Name[int(req.Platform)]
for _, fileURL := range req.FileURLs {
log := relationtb.Log{
Version: req.Version,
SystemType: req.SystemType,
Platform: platform,
UserID: userID,
CreateTime: time.Now(),
Url: fileURL.URL,
FileName: fileURL.Filename,
}
for i := 0; i < 20; i++ {
id := genLogID()
logs, err := t.thirdDatabase.GetLogs(ctx, []string{id}, "")
if err != nil {
return nil, err
}
if len(logs) == 0 {
log.LogID = id
break
}
}
if log.LogID == "" {
return nil, errs.ErrData.Wrap("Log id gen error")
}
DBlogs = append(DBlogs, &log)
}
err := t.thirdDatabase.UploadLogs(ctx, DBlogs)
if err != nil {
return nil, err
}
return &third.UploadLogsResp{}, nil
}
func (t *thirdServer) DeleteLogs(ctx context.Context, req *third.DeleteLogsReq) (*third.DeleteLogsResp, error) {
if err := authverify.CheckAdmin(ctx); err != nil {
return nil, err
}
userID := ""
logs, err := t.thirdDatabase.GetLogs(ctx, req.LogIDs, userID)
if err != nil {
return nil, err
}
var logIDs []string
for _, log := range logs {
logIDs = append(logIDs, log.LogID)
}
if ids := utils2.Single(req.LogIDs, logIDs); len(ids) > 0 {
return nil, errs.ErrRecordNotFound.Wrap(fmt.Sprintf("logIDs not found%#v", ids))
}
err = t.thirdDatabase.DeleteLogs(ctx, req.LogIDs, userID)
if err != nil {
return nil, err
}
return &third.DeleteLogsResp{}, nil
}
func dbToPbLogInfos(logs []*relationtb.Log) []*third.LogInfo {
db2pbForLogInfo := func(log *relationtb.Log) *third.LogInfo {
return &third.LogInfo{
Filename: log.FileName,
UserID: log.UserID,
Platform: utils.StringToInt32(log.Platform),
Url: log.Url,
CreateTime: log.CreateTime.UnixMilli(),
LogID: log.LogID,
SystemType: log.SystemType,
Version: log.Version,
Ex: log.Ex,
}
}
return utils.Slice(logs, db2pbForLogInfo)
}
func (t *thirdServer) SearchLogs(ctx context.Context, req *third.SearchLogsReq) (*third.SearchLogsResp, error) {
if err := authverify.CheckAdmin(ctx); err != nil {
return nil, err
}
var (
resp third.SearchLogsResp
userIDs []string
)
if req.StartTime > req.EndTime {
return nil, errs.ErrArgs.Wrap("startTime>endTime")
}
total, logs, err := t.thirdDatabase.SearchLogs(ctx, req.Keyword, time.UnixMilli(req.StartTime), time.UnixMilli(req.EndTime), req.Pagination.PageNumber, req.Pagination.ShowNumber)
if err != nil {
return nil, err
}
pbLogs := dbToPbLogInfos(logs)
for _, log := range logs {
userIDs = append(userIDs, log.UserID)
}
users, err := t.thirdDatabase.FindUsers(ctx, userIDs)
if err != nil {
return nil, err
}
IDtoName := make(map[string]string)
for _, user := range users {
IDtoName[user.UserID] = user.Nickname
}
for _, pbLog := range pbLogs {
pbLog.Nickname = IDtoName[pbLog.UserID]
}
resp.LogsInfos = pbLogs
resp.Total = total
return &resp, nil
}
+2 -1
View File
@@ -35,6 +35,7 @@ import (
"github.com/openimsdk/open-im-server/v3/pkg/common/db/controller" "github.com/openimsdk/open-im-server/v3/pkg/common/db/controller"
"github.com/openimsdk/open-im-server/v3/pkg/common/db/relation" "github.com/openimsdk/open-im-server/v3/pkg/common/db/relation"
relationtb "github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation" relationtb "github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation"
"github.com/openimsdk/open-im-server/v3/pkg/rpcclient" "github.com/openimsdk/open-im-server/v3/pkg/rpcclient"
) )
@@ -79,7 +80,7 @@ func Start(client discoveryregistry.SvcDiscoveryRegistry, server *grpc.Server) e
} }
third.RegisterThirdServer(server, &thirdServer{ third.RegisterThirdServer(server, &thirdServer{
apiURL: apiURL, apiURL: apiURL,
thirdDatabase: controller.NewThirdDatabase(cache.NewMsgCacheModel(rdb)), thirdDatabase: controller.NewThirdDatabase(cache.NewMsgCacheModel(rdb), db),
userRpcClient: rpcclient.NewUserRpcClient(client), userRpcClient: rpcclient.NewUserRpcClient(client),
s3dataBase: controller.NewS3Database(o, relation.NewObjectInfo(db)), s3dataBase: controller.NewS3Database(o, relation.NewObjectInfo(db)),
defaultExpire: time.Hour * 24 * 7, defaultExpire: time.Hour * 24 * 7,
+1
View File
@@ -29,6 +29,7 @@ type SendMsg struct {
SessionType int32 `json:"sessionType" binding:"required"` SessionType int32 `json:"sessionType" binding:"required"`
IsOnlineOnly bool `json:"isOnlineOnly"` IsOnlineOnly bool `json:"isOnlineOnly"`
NotOfflinePush bool `json:"notOfflinePush"` NotOfflinePush bool `json:"notOfflinePush"`
SendTime int64 `json:"sendTime"`
OfflinePushInfo *sdkws.OfflinePushInfo `json:"offlinePushInfo"` OfflinePushInfo *sdkws.OfflinePushInfo `json:"offlinePushInfo"`
} }
+3
View File
@@ -128,12 +128,14 @@ type configStruct struct {
SecretAccessKey string `yaml:"secretAccessKey"` SecretAccessKey string `yaml:"secretAccessKey"`
SessionToken string `yaml:"sessionToken"` SessionToken string `yaml:"sessionToken"`
SignEndpoint string `yaml:"signEndpoint"` SignEndpoint string `yaml:"signEndpoint"`
PublicRead bool `yaml:"publicRead"`
} `yaml:"minio"` } `yaml:"minio"`
Cos struct { Cos struct {
BucketURL string `yaml:"bucketURL"` BucketURL string `yaml:"bucketURL"`
SecretID string `yaml:"secretID"` SecretID string `yaml:"secretID"`
SecretKey string `yaml:"secretKey"` SecretKey string `yaml:"secretKey"`
SessionToken string `yaml:"sessionToken"` SessionToken string `yaml:"sessionToken"`
PublicRead bool `yaml:"publicRead"`
} `yaml:"cos"` } `yaml:"cos"`
Oss struct { Oss struct {
Endpoint string `yaml:"endpoint"` Endpoint string `yaml:"endpoint"`
@@ -142,6 +144,7 @@ type configStruct struct {
AccessKeyID string `yaml:"accessKeyID"` AccessKeyID string `yaml:"accessKeyID"`
AccessKeySecret string `yaml:"accessKeySecret"` AccessKeySecret string `yaml:"accessKeySecret"`
SessionToken string `yaml:"sessionToken"` SessionToken string `yaml:"sessionToken"`
PublicRead bool `yaml:"publicRead"`
} `yaml:"oss"` } `yaml:"oss"`
} `yaml:"object"` } `yaml:"object"`
+6 -13
View File
@@ -59,11 +59,8 @@ type ConversationCache interface {
DelConversations(ownerUserID string, conversationIDs ...string) ConversationCache DelConversations(ownerUserID string, conversationIDs ...string) ConversationCache
DelUsersConversation(conversationID string, ownerUserIDs ...string) ConversationCache DelUsersConversation(conversationID string, ownerUserIDs ...string) ConversationCache
// get one conversation from msgCache // get one conversation from msgCache
GetConversations( GetConversations(ctx context.Context, ownerUserID string,
ctx context.Context, conversationIDs []string) ([]*relationtb.ConversationModel, error)
ownerUserID string,
conversationIDs []string,
) ([]*relationtb.ConversationModel, error)
// get one user's all conversations from msgCache // get one user's all conversations from msgCache
GetUserAllConversations(ctx context.Context, ownerUserID string) ([]*relationtb.ConversationModel, error) GetUserAllConversations(ctx context.Context, ownerUserID string) ([]*relationtb.ConversationModel, error)
// get user conversation recv msg from msgCache // get user conversation recv msg from msgCache
@@ -79,10 +76,8 @@ type ConversationCache interface {
GetUserAllHasReadSeqs(ctx context.Context, ownerUserID string) (map[string]int64, error) GetUserAllHasReadSeqs(ctx context.Context, ownerUserID string) (map[string]int64, error)
DelUserAllHasReadSeqs(ownerUserID string, conversationIDs ...string) ConversationCache DelUserAllHasReadSeqs(ownerUserID string, conversationIDs ...string) ConversationCache
GetConversationsByConversationID( GetConversationsByConversationID(ctx context.Context,
ctx context.Context, conversationIDs []string) ([]*relationtb.ConversationModel, error)
conversationIDs []string,
) ([]*relationtb.ConversationModel, error)
DelConversationByConversationID(conversationIDs ...string) ConversationCache DelConversationByConversationID(conversationIDs ...string) ConversationCache
GetConversationNotReceiveMessageUserIDs(ctx context.Context, conversationID string) ([]string, error) GetConversationNotReceiveMessageUserIDs(ctx context.Context, conversationID string) ([]string, error)
DelConversationNotReceiveMessageUserIDs(conversationIDs ...string) ConversationCache DelConversationNotReceiveMessageUserIDs(conversationIDs ...string) ConversationCache
@@ -418,10 +413,8 @@ func (c *ConversationRedisCache) GetUserAllHasReadSeqs(
) )
} }
func (c *ConversationRedisCache) DelUserAllHasReadSeqs( func (c *ConversationRedisCache) DelUserAllHasReadSeqs(ownerUserID string,
ownerUserID string, conversationIDs ...string) ConversationCache {
conversationIDs ...string,
) ConversationCache {
cache := c.NewCache() cache := c.NewCache()
for _, conversationID := range conversationIDs { for _, conversationID := range conversationIDs {
cache.AddKeys(c.getConversationHasReadSeqKey(ownerUserID, conversationID)) cache.AddKeys(c.getConversationHasReadSeqKey(ownerUserID, conversationID))
+42 -3
View File
@@ -16,21 +16,60 @@ package controller
import ( import (
"context" "context"
"time"
"gorm.io/gorm"
"github.com/openimsdk/open-im-server/v3/pkg/common/db/cache" "github.com/openimsdk/open-im-server/v3/pkg/common/db/cache"
dbimpl "github.com/openimsdk/open-im-server/v3/pkg/common/db/relation"
"github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation"
) )
type ThirdDatabase interface { type ThirdDatabase interface {
FcmUpdateToken(ctx context.Context, account string, platformID int, fcmToken string, expireTime int64) error FcmUpdateToken(ctx context.Context, account string, platformID int, fcmToken string, expireTime int64) error
SetAppBadge(ctx context.Context, userID string, value int) error SetAppBadge(ctx context.Context, userID string, value int) error
//about log for debug
UploadLogs(ctx context.Context, logs []*relation.Log) error
DeleteLogs(ctx context.Context, logID []string, userID string) error
SearchLogs(ctx context.Context, keyword string, start time.Time, end time.Time, pageNumber int32, showNumber int32) (uint32, []*relation.Log, error)
GetLogs(ctx context.Context, LogIDs []string, userID string) ([]*relation.Log, error)
FindUsers(ctx context.Context, userIDs []string) ([]*relation.UserModel, error)
} }
type thirdDatabase struct { type thirdDatabase struct {
cache cache.MsgModel cache cache.MsgModel
logdb relation.LogInterface
userdb relation.UserModelInterface
} }
func NewThirdDatabase(cache cache.MsgModel) ThirdDatabase { // FindUsers implements ThirdDatabase.
return &thirdDatabase{cache: cache} func (t *thirdDatabase) FindUsers(ctx context.Context, userIDs []string) ([]*relation.UserModel, error) {
return t.userdb.Find(ctx, userIDs)
}
// DeleteLogs implements ThirdDatabase.
func (t *thirdDatabase) DeleteLogs(ctx context.Context, logID []string, userID string) error {
return t.logdb.Delete(ctx, logID, userID)
}
// GetLogs implements ThirdDatabase.
func (t *thirdDatabase) GetLogs(ctx context.Context, LogIDs []string, userID string) ([]*relation.Log, error) {
return t.logdb.Get(ctx, LogIDs, userID)
}
// SearchLogs implements ThirdDatabase.
func (t *thirdDatabase) SearchLogs(ctx context.Context, keyword string, start time.Time, end time.Time, pageNumber int32, showNumber int32) (uint32, []*relation.Log, error) {
return t.logdb.Search(ctx, keyword, start, end, pageNumber, showNumber)
}
// UploadLogs implements ThirdDatabase.
func (t *thirdDatabase) UploadLogs(ctx context.Context, logs []*relation.Log) error {
return t.logdb.Create(ctx, logs)
}
func NewThirdDatabase(cache cache.MsgModel, db *gorm.DB) ThirdDatabase {
return &thirdDatabase{cache: cache, logdb: dbimpl.NewLogGorm(db), userdb: dbimpl.NewUserGorm(db)}
} }
func (t *thirdDatabase) FcmUpdateToken( func (t *thirdDatabase) FcmUpdateToken(
+47
View File
@@ -0,0 +1,47 @@
package relation
import (
"context"
"time"
"github.com/OpenIMSDK/tools/errs"
"github.com/OpenIMSDK/tools/ormutil"
"gorm.io/gorm"
relationtb "github.com/openimsdk/open-im-server/v3/pkg/common/db/table/relation"
)
type LogGorm struct {
db *gorm.DB
}
func (l *LogGorm) Create(ctx context.Context, log []*relationtb.Log) error {
return errs.Wrap(l.db.WithContext(ctx).Create(log).Error)
}
func (l *LogGorm) Search(ctx context.Context, keyword string, start time.Time, end time.Time, pageNumber int32, showNumber int32) (uint32, []*relationtb.Log, error) {
db := l.db.WithContext(ctx).Where("create_time >= ?", start)
if end.UnixMilli() != 0 {
db = l.db.WithContext(ctx).Where("create_time <= ?", end)
}
return ormutil.GormSearch[relationtb.Log](db, []string{"user_id"}, keyword, pageNumber, showNumber)
}
func (l *LogGorm) Delete(ctx context.Context, logIDs []string, userID string) error {
if userID == "" {
return errs.Wrap(l.db.WithContext(ctx).Where("log_id in ?", logIDs).Delete(&relationtb.Log{}).Error)
}
return errs.Wrap(l.db.WithContext(ctx).Where("log_id in ? and user_id=?", logIDs, userID).Delete(&relationtb.Log{}).Error)
}
func (l *LogGorm) Get(ctx context.Context, logIDs []string, userID string) ([]*relationtb.Log, error) {
var logs []*relationtb.Log
if userID == "" {
return logs, errs.Wrap(l.db.WithContext(ctx).Where("log_id in ?", logIDs).Find(&logs).Error)
}
return logs, errs.Wrap(l.db.WithContext(ctx).Where("log_id in ? and user_id=?", logIDs, userID).Find(&logs).Error)
}
func NewLogGorm(db *gorm.DB) relationtb.LogInterface {
db.AutoMigrate(&relationtb.Log{})
return &LogGorm{db: db}
}
+15 -5
View File
@@ -288,7 +288,7 @@ func (c *Cos) AccessURL(ctx context.Context, name string, expire time.Duration,
style = append(style, "format/"+opt.Image.Format) style = append(style, "format/"+opt.Image.Format)
} }
if len(style) > 0 { if len(style) > 0 {
imageMogr = "&imageMogr2/thumbnail/" + strings.Join(style, "/") + "/ignore-error/1" imageMogr = "imageMogr2/thumbnail/" + strings.Join(style, "/") + "/ignore-error/1"
} }
} }
if opt.ContentType != "" { if opt.ContentType != "" {
@@ -306,13 +306,23 @@ func (c *Cos) AccessURL(ctx context.Context, name string, expire time.Duration,
} else if expire < time.Second { } else if expire < time.Second {
expire = time.Second expire = time.Second
} }
rawURL, err := c.client.Object.GetPresignedURL(ctx, http.MethodGet, name, c.credential.SecretID, c.credential.SecretKey, expire, &option) rawURL, err := c.getPresignedURL(ctx, name, expire, &option)
if err != nil { if err != nil {
return "", err return "", err
} }
urlStr := rawURL.String()
if imageMogr != "" { if imageMogr != "" {
urlStr += imageMogr if rawURL.RawQuery == "" {
rawURL.RawQuery = imageMogr
} else {
rawURL.RawQuery = rawURL.RawQuery + "&" + imageMogr
}
} }
return urlStr, nil return rawURL.String(), nil
}
func (c *Cos) getPresignedURL(ctx context.Context, name string, expire time.Duration, opt *cos.PresignedURLOptions) (*url.URL, error) {
if !config.Config.Object.Cos.PublicRead {
return c.client.Object.GetPresignedURL(ctx, http.MethodGet, name, c.credential.SecretID, c.credential.SecretKey, expire, opt)
}
return c.client.Object.GetObjectURL(name), nil
} }
-15
View File
@@ -1,15 +0,0 @@
// Copyright © 2023 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cos // import "github.com/openimsdk/open-im-server/v3/pkg/common/db/s3/cos"
+13
View File
@@ -0,0 +1,13 @@
package cos
import (
"context"
"net/http"
"net/url"
_ "unsafe"
"github.com/tencentyun/cos-go-sdk-v5"
)
//go:linkname newRequest github.com/tencentyun/cos-go-sdk-v5.(*Client).newRequest
func newRequest(c *cos.Client, ctx context.Context, baseURL *url.URL, uri, method string, body interface{}, optQuery interface{}, optHeader interface{}) (req *http.Request, err error)
-15
View File
@@ -1,15 +0,0 @@
// Copyright © 2023 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package minio // import "github.com/openimsdk/open-im-server/v3/pkg/common/db/s3/minio"
+11
View File
@@ -0,0 +1,11 @@
package minio
import (
"net/url"
_ "unsafe"
"github.com/minio/minio-go/v7"
)
//go:linkname makeTargetURL github.com/minio/minio-go/v7.(*Client).makeTargetURL
func makeTargetURL(client *minio.Client, bucketName, objectName, bucketLocation string, isVirtualHostStyle bool, queryValues url.Values) (*url.URL, error)
+18 -1
View File
@@ -139,6 +139,15 @@ func (m *Minio) initMinio(ctx context.Context) error {
return fmt.Errorf("make bucket error: %w", err) return fmt.Errorf("make bucket error: %w", err)
} }
} }
if conf.PublicRead {
policy := fmt.Sprintf(
`{"Version": "2012-10-17","Statement": [{"Action": ["s3:GetObject","s3:PutObject"],"Effect": "Allow","Principal": {"AWS": ["*"]},"Resource": ["arn:aws:s3:::%s/*"],"Sid": ""}]}`,
conf.Bucket,
)
if err := m.core.Client.SetBucketPolicy(ctx, conf.Bucket, policy); err != nil {
return err
}
}
m.location, err = m.core.Client.GetBucketLocation(ctx, conf.Bucket) m.location, err = m.core.Client.GetBucketLocation(ctx, conf.Bucket)
if err != nil { if err != nil {
return err return err
@@ -375,7 +384,15 @@ func (m *Minio) presignedGetObject(ctx context.Context, name string, expire time
} else if expire < time.Second { } else if expire < time.Second {
expire = time.Second expire = time.Second
} }
rawURL, err := m.sign.PresignedGetObject(ctx, m.bucket, name, expire, query) var (
rawURL *url.URL
err error
)
if config.Config.Object.Minio.PublicRead {
rawURL, err = makeTargetURL(m.sign, m.bucket, name, m.location, false, query)
} else {
rawURL, err = m.sign.PresignedGetObject(ctx, m.bucket, name, expire, query)
}
if err != nil { if err != nil {
return "", err return "", err
} }
-15
View File
@@ -1,15 +0,0 @@
// Copyright © 2023 OpenIM. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package oss // import "github.com/openimsdk/open-im-server/v3/pkg/common/db/s3/oss"
@@ -16,10 +16,24 @@ package oss
import ( import (
"net/http" "net/http"
"net/url"
_ "unsafe" _ "unsafe"
"github.com/aliyun/aliyun-oss-go-sdk/oss" "github.com/aliyun/aliyun-oss-go-sdk/oss"
) )
//go:linkname ossSignHeader github.com/aliyun/aliyun-oss-go-sdk/oss.(*Conn).signHeader //go:linkname signHeader github.com/aliyun/aliyun-oss-go-sdk/oss.Conn.signHeader
func ossSignHeader(c *oss.Conn, req *http.Request, canonicalizedResource string) func signHeader(c oss.Conn, req *http.Request, canonicalizedResource string)
//go:linkname getURLParams github.com/aliyun/aliyun-oss-go-sdk/oss.Conn.getURLParams
func getURLParams(c oss.Conn, params map[string]interface{}) string
//go:linkname getURL github.com/aliyun/aliyun-oss-go-sdk/oss.urlMaker.getURL
func getURL(um urlMaker, bucket, object, params string) *url.URL
type urlMaker struct {
Scheme string
NetLoc string
Type int
IsProxy bool
}
+21 -7
View File
@@ -20,6 +20,7 @@ import (
"fmt" "fmt"
"net/http" "net/http"
"net/url" "net/url"
"reflect"
"strconv" "strconv"
"strings" "strings"
"time" "time"
@@ -69,6 +70,7 @@ func NewOSS() (s3.Interface, error) {
bucketURL: conf.BucketURL, bucketURL: conf.BucketURL,
bucket: bucket, bucket: bucket,
credentials: client.Config.GetCredentials(), credentials: client.Config.GetCredentials(),
um: *(*urlMaker)(reflect.ValueOf(bucket.Client.Conn).Elem().FieldByName("url").UnsafePointer()),
}, nil }, nil
} }
@@ -76,6 +78,7 @@ type OSS struct {
bucketURL string bucketURL string
bucket *oss.Bucket bucket *oss.Bucket
credentials oss.Credentials credentials oss.Credentials
um urlMaker
} }
func (o *OSS) Engine() string { func (o *OSS) Engine() string {
@@ -163,7 +166,7 @@ func (o *OSS) AuthSign(ctx context.Context, uploadID string, name string, expire
request.Header.Set(oss.HTTPHeaderHost, request.Host) request.Header.Set(oss.HTTPHeaderHost, request.Host)
request.Header.Set(oss.HTTPHeaderDate, now) request.Header.Set(oss.HTTPHeaderDate, now)
request.Header.Set(oss.HttpHeaderOssDate, now) request.Header.Set(oss.HttpHeaderOssDate, now)
ossSignHeader(o.bucket.Client.Conn, request, fmt.Sprintf(`/%s/%s?partNumber=%d&uploadId=%s`, o.bucket.BucketName, name, partNumber, uploadID)) signHeader(*o.bucket.Client.Conn, request, fmt.Sprintf(`/%s/%s?partNumber=%d&uploadId=%s`, o.bucket.BucketName, name, partNumber, uploadID))
delete(request.Header, oss.HTTPHeaderDate) delete(request.Header, oss.HTTPHeaderDate)
result.Parts[i] = s3.SignPart{ result.Parts[i] = s3.SignPart{
PartNumber: partNumber, PartNumber: partNumber,
@@ -272,6 +275,7 @@ func (o *OSS) ListUploadedParts(ctx context.Context, uploadID string, name strin
} }
func (o *OSS) AccessURL(ctx context.Context, name string, expire time.Duration, opt *s3.AccessURLOption) (string, error) { func (o *OSS) AccessURL(ctx context.Context, name string, expire time.Duration, opt *s3.AccessURLOption) (string, error) {
publicRead := config.Config.Object.Oss.PublicRead
var opts []oss.Option var opts []oss.Option
if opt != nil { if opt != nil {
if opt.Image != nil { if opt.Image != nil {
@@ -299,11 +303,13 @@ func (o *OSS) AccessURL(ctx context.Context, name string, expire time.Duration,
process += ",format," + format process += ",format," + format
opts = append(opts, oss.Process(process)) opts = append(opts, oss.Process(process))
} }
if opt.ContentType != "" { if !publicRead {
opts = append(opts, oss.ResponseContentType(opt.ContentType)) if opt.ContentType != "" {
} opts = append(opts, oss.ResponseContentType(opt.ContentType))
if opt.Filename != "" { }
opts = append(opts, oss.ResponseContentDisposition(`attachment; filename=`+strconv.Quote(opt.Filename))) if opt.Filename != "" {
opts = append(opts, oss.ResponseContentDisposition(`attachment; filename=`+strconv.Quote(opt.Filename)))
}
} }
} }
if expire <= 0 { if expire <= 0 {
@@ -311,5 +317,13 @@ func (o *OSS) AccessURL(ctx context.Context, name string, expire time.Duration,
} else if expire < time.Second { } else if expire < time.Second {
expire = time.Second expire = time.Second
} }
return o.bucket.SignURL(name, http.MethodGet, int64(expire/time.Second), opts...) if !publicRead {
return o.bucket.SignURL(name, http.MethodGet, int64(expire/time.Second), opts...)
}
rawParams, err := oss.GetRawParams(opts)
if err != nil {
return "", err
}
params := getURLParams(*o.bucket.Client.Conn, rawParams)
return getURL(o.um, o.bucket.BucketName, name, params).String(), nil
} }
+25
View File
@@ -0,0 +1,25 @@
package relation
import (
"context"
"time"
)
type Log struct {
LogID string `gorm:"column:log_id;primary_key;type:char(64)"`
Platform string `gorm:"column:platform;type:varchar(32)"`
UserID string `gorm:"column:user_id;type:char(64)"`
CreateTime time.Time `gorm:"index:,sort:desc"`
Url string `gorm:"column:url;type varchar(255)"`
FileName string `gorm:"column:filename;type varchar(255)"`
SystemType string `gorm:"column:system_type;type varchar(255)"`
Version string `gorm:"column:version;type varchar(255)"`
Ex string `gorm:"column:ex;type varchar(255)"`
}
type LogInterface interface {
Create(ctx context.Context, log []*Log) error
Search(ctx context.Context, keyword string, start time.Time, end time.Time, pageNumber int32, showNumber int32) (uint32, []*Log, error)
Delete(ctx context.Context, logID []string, userID string) error
Get(ctx context.Context, logIDs []string, userID string) ([]*Log, error)
}
+4
View File
@@ -186,17 +186,21 @@ def "MINIO_ACCESS_KEY" "${USER}"
def "MINIO_SECRET_KEY" "${PASSWORD}" # MinIO的密钥 def "MINIO_SECRET_KEY" "${PASSWORD}" # MinIO的密钥
def "MINIO_SESSION_TOKEN" # MinIO的会话令牌 def "MINIO_SESSION_TOKEN" # MinIO的会话令牌
readonly MINIO_SIGN_ENDPOINT=${MINIO_SIGN_ENDPOINT:-"http://${IP}:${MINIO_PORT}"} # signEndpoint为minio公网地址 readonly MINIO_SIGN_ENDPOINT=${MINIO_SIGN_ENDPOINT:-"http://${IP}:${MINIO_PORT}"} # signEndpoint为minio公网地址
def "MINIO_PUBLIC_READ" "false" # 公有读
# 腾讯云COS的存储桶URL # 腾讯云COS的存储桶URL
def "COS_BUCKET_URL" "https://temp-1252357374.cos.ap-chengdu.myqcloud.com" def "COS_BUCKET_URL" "https://temp-1252357374.cos.ap-chengdu.myqcloud.com"
def "COS_SECRET_ID" # 腾讯云COS的密钥ID def "COS_SECRET_ID" # 腾讯云COS的密钥ID
def "COS_SECRET_KEY" # 腾讯云COS的密钥 def "COS_SECRET_KEY" # 腾讯云COS的密钥
def "COS_SESSION_TOKEN" # 腾讯云COS的会话令牌 def "COS_SESSION_TOKEN" # 腾讯云COS的会话令牌
def "COS_PUBLIC_READ" "false" # 公有读
def "OSS_ENDPOINT" "https://oss-cn-chengdu.aliyuncs.com" # 阿里云OSS的端点URL def "OSS_ENDPOINT" "https://oss-cn-chengdu.aliyuncs.com" # 阿里云OSS的端点URL
def "OSS_BUCKET" "demo-9999999" # 阿里云OSS的存储桶名称 def "OSS_BUCKET" "demo-9999999" # 阿里云OSS的存储桶名称
def "OSS_BUCKET_URL" "https://demo-9999999.oss-cn-chengdu.aliyuncs.com" # 阿里云OSS的存储桶URL def "OSS_BUCKET_URL" "https://demo-9999999.oss-cn-chengdu.aliyuncs.com" # 阿里云OSS的存储桶URL
def "OSS_ACCESS_KEY_ID" # 阿里云OSS的访问密钥ID def "OSS_ACCESS_KEY_ID" # 阿里云OSS的访问密钥ID
def "OSS_ACCESS_KEY_SECRET" # 阿里云OSS的密钥 def "OSS_ACCESS_KEY_SECRET" # 阿里云OSS的密钥
def "OSS_SESSION_TOKEN" # 阿里云OSS的会话令牌 def "OSS_SESSION_TOKEN" # 阿里云OSS的会话令牌
def "OSS_PUBLIC_READ" "false" # 公有读
###################### Redis 配置信息 ###################### ###################### Redis 配置信息 ######################
def "REDIS_PORT" "16379" # Redis的端口 def "REDIS_PORT" "16379" # Redis的端口
+19 -12
View File
@@ -22,7 +22,6 @@ import (
"net" "net"
"net/url" "net/url"
"os" "os"
"strings"
"time" "time"
"github.com/minio/minio-go/v7" "github.com/minio/minio-go/v7"
@@ -117,16 +116,24 @@ func main() {
os.Exit(1) os.Exit(1)
} }
func exactIP(urll string) string { func checkMinioIP() error {
u, _ := url.Parse(urll) for _, i := range []string{config.Config.Object.ApiURL, config.Config.Object.Minio.SignEndpoint} {
host, _, err := net.SplitHostPort(u.Host) u, err := url.Parse(i)
if err != nil { if err != nil {
host = u.Host return utils.Wrap(err, "api format error,please check config file apiURL or Minio SignEndpoint")
}
if u.Scheme == "https" {
continue
}
host, _, err := net.SplitHostPort(u.Host)
if err != nil {
host = u.Host
}
if host == "127.0.0.1" {
return ErrConfig.Wrap("apiURL or Minio SignEndpoint endpoint contain 127.0.0.1,please modify it")
}
} }
if strings.HasSuffix(host, ":") { return nil
host = host[0 : len(host)-1]
}
return host
} }
func checkMysql() error { func checkMysql() error {
@@ -205,8 +212,8 @@ func checkMinio() error {
return ErrComponentStart.Wrap("Minio server is offline") return ErrComponentStart.Wrap("Minio server is offline")
} }
} }
if exactIP(config.Config.Object.ApiURL) == "127.0.0.1" || exactIP(config.Config.Object.Minio.SignEndpoint) == "127.0.0.1" { if checkMinioIP() != nil {
return ErrConfig.Wrap("apiURL or Minio SignEndpoint endpoint contain 127.0.0.1") return checkMinioIP()
} }
} }
return nil return nil
+3 -2
View File
@@ -1,4 +1,4 @@
module github.com/openimsdk/open-im-server/v3/tools/imctl module github.com/openimsdk/open-im-server/v3/tools/imctl
go 1.18 go 1.18
@@ -14,5 +14,6 @@ require (
require ( require (
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect
golang.org/x/sys v0.1.0 // indirect golang.org/x/sys v0.10.0 // indirect
k8s.io/kubernetes v1.28.2
) )
+3
View File
@@ -20,5 +20,8 @@ github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U= golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
k8s.io/kubernetes v1.28.2 h1:GhcnYeNTukeaC0dD5BC+UWBvzQsFEpWj7XBVMQptfYc=
k8s.io/kubernetes v1.28.2/go.mod h1:FmB1Mlp9ua0ezuwQCTGs/y6wj/fVisN2sVxhzjj0WDk=