Skip to content

Commit

Permalink
Merge pull request #3 from uavorg/master
Browse files Browse the repository at this point in the history
Merge pull request #3 from uavorg/master
  • Loading branch information
zhiqiuxie authored Jun 7, 2018
2 parents 8b88e90 + 166c0c6 commit 0874d5f
Show file tree
Hide file tree
Showing 209 changed files with 8,165 additions and 2,598 deletions.
178 changes: 78 additions & 100 deletions README.md

Large diffs are not rendered by default.

173 changes: 173 additions & 0 deletions README中文版.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
# UAVStack
UAVStack是智能化服务技术栈,是研发运维一体化的解决方案。UAV是无人机的缩写,寓意无人机翱翔蓝天,智能的,透明的完成任务。
它包括任务机器人(代号HIT),全维监控(代号UAV.Monitor),应用性能管理(代号UAV.APM),服务治理(代号UAV.ServiceGovern),微服务计算(代号UAV.MSCP),用户体验管理(代号UAV.UEM)等。

UAVStack is an intelligent service technology stack. It is a solution for integration of R & D, operation and maintenance. UAV is the abbreviation of unmanned aerial vehicle (UAV), which means that unmanned aerial vehicle flies into the sky, and completes tasks intelligently and transparently.

It includes AIOps Robot (HIT),full dimension monitoring (UAV.Monitor), application performance management (UAV.APM), service governance (UAV.ServiceGovern), micro-service computing (UAV.MSCP), user experience management (UAV.UEM) etc..


目前UAVStack开源系列 ( 官方网站:https://uavorg.github.io/main/ ) 包括

UAVStack Open Source series includes UAV.Monitor, UAV.APM, UAV.ServiceGovern, UAV.MSCP

![Uavstack开源系列](https://uavorg.github.io/main/index/img/support/openSource.png)

# 更新模式 Project Updating Principle
* 开源系列源代码Issue更新,将与内部版本完全同步,最低频率每周一次
* 开源系列源代码相关的功能优化,将与内部版本完全同步,最低频率每周一次
* 开源系列源代码相关的新功能开放,会提前一周预告,并于下周内发布
* 标准部署包下载会与源代码更新同步,最低频率每周一次
* AllInOne开发演示版原则上每月更新一次(打包上传比较耗时),中间更新请下载标准部署包替换
---
* The open source series updating is fully synchronized with the internal version, with the lowest frequency once a week
* The open source series related functional optimizations is fully synchronized with the internal version, with the lowest frequency once a week
* The open source series related new features will be announced a week ahead of schedule, and released next week
* The standard deployment pack download is synchronized with source code updates, with the lowest frequency once a week
* AllInOne pack are updated once a month (packaged, uploaded, and time-consuming) in principle, and you can download the standard deployment package instead


# 工程说明 Source Code Project Description
uavstack是UAVStack的All in One源代码仓库。

uavstack is the All in One source repository of UAVStack.

## 根POM
* com.creditease.uav.superpom 根POM,包含依赖管理,Build所需plugins等 (Parent POM for dependency management & build plugins)

## UAV.Monitor
* com.creditease.uav.agent 监控代理程序MonitorAgent核心框架 (Core of MonitorAgent)
* com.creditease.uav.healthmanager Monitor核心服务,包括实时数据,画像数据存储/查询,实时报警流式计算,AppHub支持服务 (Core feature of HealthManager, including Monitoring Data Service,Application/Service Profiling,Real-time alarm streaming,AppHub Support Service)
* com.creditease.uav.notifycenter Monitor服务:报警中心服务端 (Notification Center Service)
* com.creditease.uav.monitorframework 探针MOF核心框架 (Core of JVM MonitorFramework for Tomcat,Jetty,SpringBoot,MSCP, etc...)
* com.creditease.uav.monitorframework.agent 探针MOF的javaagent实现 (JVM MonitorFramework javaagent support)
* com.creditease.uav.monitorframework.dproxy javaassit支持 (JVM MonitorFramework javassit support)
* com.creditease.uav.hook.* 探针MOF针对技术规范的钩子,包括Dubbo,HttpClients,JAXWS,JDBC,MongoClients,MQ,Redis等 (JVM MonitorFramework Hooks for application framework)
* com.creditease.uav.loghook 探针MOF针对log4j,logback支持 (JVM MonitorFramework Hook for log4j, logback)
* com.creditease.uav.tomcat.plus.core 探针MOF对Tomcat/SpringBoot的支持 (JVM MonitorFramework extension for Tomcat/SpringBoot)
* com.creditease.uav.jetty.plus.core 探针MOF对Jetty支持 (JVM MonitorFramework extension for Jetty)
* com.creditease.uav.mock.sl4j 去除某些框架依赖sl4j (mock sl4j)
* com.creditease.uav.ttl com.alibaba.ttl源代码引入 (Alibaba TTL)
* com.creditease.uav.monitorframework.buildFat WAR测试程序 (Function Test of JVM MonitorFramework)
* com.creditease.uav.monitorframework.springbootFat SpringBoot测试程序 (Function Test of JVM MonitorFramework for springboot)
* com.creditease.uav.agent.buildComponent 制作监控代理程序部署包 (Build POM for MonitorAgent)
* com.creditease.uav.healthmanager.buildComponent 制作健康管理程序部署包 (Build POM for HealthManager)
* com.creditease.uav.monitorframework.buildComponent 制作探针MOF部署包 (Build POM for JVM MonitorFramework)

## UAV.APM
* com.creditease.uav.collect APM归集客户端/服务端 (Core of APM Data Collection Client & Service)
* com.creditease.uav.invokechain 调用链,日志服务存储和查询 (InvokeChain, Log Collection 2.0)
* com.creditease.uav.monitorframework.apm 探针MOF的APM支持 (JVM MonitorFramework extension for APM)
* com.creditease.uav.threadanalysis 一键式线程分析客户端和服务端 (One Shot Thread Analysis Client & Service)

## UAV.MSCP
* com.creditease.uav.base MSCP核心框架 (Core of MSCP)
* com.creditease.uav.agent.heartbeat 心跳客户端,心跳服务端,节点远程操控,进程扫描,进程值守 (Base feature of MSCP including heartbeat client & service,node remote opertaions,process auto scanning,process keep-alive support)
* com.creditease.uav.annoscan Fastclasspathscanner源代码引入,注解Class扫描 (Fastclasspathscanner)
* com.creditease.uav.cache.redis 基于Redis的CacheManager (Cache Framework based on redis)
* com.creditease.uav.dbaccess 存储管理器,目前封装了对MySql,OpenTSDB,HBase,MongoDB的存储和查询 (Data Store Framework for MySQL,OpenTSDB,HBase,MongoDB)
* com.creditease.uav.elasticsearch.client ElasticSearch的Shaded客户端封装,避免冲突 (ElasticSearch-Shaded-Client)
* com.creditease.uav.fastjson Fastjson源代码引入 (FastJson)
* com.creditease.uav.helper Util型支持类库 (Util Support Lib)
* com.creditease.uav.httpasync Http异步通信客户端,封装Apache AsyncClient (RPC Framework based on Apache Http AsyncClient)
* com.creditease.uav.logging MSCP日志支持 (MSCP Logging support)
* com.creditease.uav.messaging MSCP消息发送者和消费者支持,无需关心底层消息服务,依赖com.creditease.uav.mq (Messaging Service)
* com.creditease.uav.mq 消息队列服务底层封装,目前使用RocketMQ (Message Queue Support based on Rocket MQ)
* com.creditease.uav.notifymanager MSCP组件级报警支持 (MSCP Component level notification support)
* com.creditease.uav.upgrade MSCP升级客户端和服务端 (Remote upgrading support for MSCP)

## UAV.AppHub
* com.creditease.uav.console UAVStack交互前端,包括Monitor,APM等前端交互功能 (AppHub is the GUI console for UAVStack)
* com.creditease.uav.console.buildComponent 制作AppHub部署包 (Build POM for AppHub)

## UAV.ServiceGovern
* 服务自动注册包含与探针MOF代码中
* 服务发现代码包含与健康管理程序画像数据查询中
* 服务降级保护(代码整理中,陆续更新...)
* 服务授权(代码整理中,陆续更新...)

# 如何构建 How to Build
如果由于缺少jar包或某些依赖jar无法下载导致Maven Build Failure,可下载[参考Maven依赖仓库](https://pan.baidu.com/s/1gfeY8pH),使用该Maven仓库或将其复制到你正在使用的Maven仓库中。

If you are unable to build because of the lack of jar packages or certain dependency jar, pleaser refer this [Maven Repository](https://pan.baidu.com/s/1gfeY8pH), use the Maven repository, or copy it to the Maven warehouse you are using.

1. build com.creditease.uav.superpom
```
cd com.creditease.uav.superpom
mvn clean install
```
2 中间件增强框架(MOF探针)
```
cd com.creditease.uav.monitorframework.buildComponent
mvn clean install
```

build结果
>target
> -build
> -uavmof_1.0_XXXXX.zip
> -uavmof_1.0_XXXXX.zip.MD5
3. 监控代理程序(MA)
```
cd com.creditease.uav.agent.buildComponent
mvn clean install
```

build结果
>target
> -build
> -uavagent_1.0_XXXXX.zip
> -uavagent_1.0_XXXXX.zip.MD5
4. 健康管理服务(HM)
```
cd com.creditease.uav.healthmanager.buildComponent
mvn clean install
```

build结果
>target
> -build
> -uavhm_1.0_XXXXX.zip
> -uavhm_1.0_XXXXX.zip.MD5
5. AppHub
```
cd com.creditease.uav.console
mvn clean install
```

build结果
>target
> -com.creditease.uav.console-1.0.war
# 文档中心 Documents
## 全维监控UAV.Monitor+APM
* [用户指南](https://uavorg.github.io/documents/uavdoc_useroperation/index.html)
* [安装部署](https://uavorg.github.io/documents/uavdoc_deploydocs/index.html)
* [架构说明](https://uavorg.github.io/documents/uavdoc_architecture/index.html)

# 下载中心 Downloads
## 全维监控UAV.Monitor+APM
* [MOF探针](https://pan.baidu.com/s/1cg4J0q)
* [监控代理程序](https://pan.baidu.com/s/1ge5MJ9h)
* [健康管理服务](https://pan.baidu.com/s/1i4HnV85)
* [AppHub](https://pan.baidu.com/s/1dFxtDZV)

## 第三方下载 Thirdparty Downloads
* [rocket.war](https://pan.baidu.com/s/1dF6NeHN)

## AllInOne开发演示版 AllInOne Install Downloads
开发演示版的健康管理服务(HM)仅适合开发环境,演示环境,小规模测试环境。
生产环境推荐使用[分布式部署](https://uavorg.github.io/documents/uavdoc_deploydocs/healmanagerInstall/healmanagerInstall/microservice.html)

The health management service (HM) in AllInOne Install Package is only suitable for development environments, demos, and small scale testing environments. [A distributed deployment](https://uavorg.github.io/documents/uavdoc_deploydocs/healmanagerInstall/healmanagerInstall/microservice.html) is recommended for the production environment

* [Windows64位](https://pan.baidu.com/s/1jIF0wNs)
* [Mac](https://pan.baidu.com/s/1mhCykp6)
* [Linux(CentOS)](https://pan.baidu.com/s/1nvj6jW1)




2 changes: 1 addition & 1 deletion com.creditease.uav.agent.buildComponent/bin/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ export CLASSPATH=bin/com.creditease.uav.base-1.0-boot.jar
echo $CLASSPATH
javaAgent="-javaagent:../uavmof/com.creditease.uav.agent/com.creditease.uav.monitorframework.agent-1.0-agent.jar"
javaOpts="-server -Xms64m -Xmx256m -Xss256k -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:CMSIncrementalDutyCycleMin=0 -XX:CMSIncrementalDutyCycle=10 -XX:+UseParNewGC -XX:+UseCMSCompactAtFullCollection -XX:-CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=0 -XX:CMSInitiatingOccupancyFraction=70 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=."
$executeJava $javaAgent $javaOpts -XX:OnOutOfMemoryError='kill -9 %p' -DNetCardIndex=$netcardIndex -DNetCardName=$netcardName -DJAppID=$2 -DJAppGroup=UNKNOWN -classpath $CLASSPATH com.creditease.mscp.boot.MSCPBoot -p $1 &
$executeJava $javaAgent $javaOpts -XX:OnOutOfMemoryError='kill -9 %p' -DNetCardIndex=$netcardIndex -DNetCardName=$netcardName -DJAppID=$2 -DJAppGroup=UAV -classpath $CLASSPATH com.creditease.mscp.boot.MSCPBoot -p $1 &

# add crontab process watcher
if [ "$proc_watcher" == "yes" ]; then
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ feature.monitoragent.http.enable=false
feature.monitoragent.jvm.filters=com.sun.tools,sun.tools,jenkins-cli.jar,com.creditease.agent.feature.nodeopagent.NodeOperCtrlClient
#monitoragent detect interval default is 30000
feature.monitoragent.detector.interval=30000
feature.monitoragent.detector.profilehbtimeout=15000
#enable local scan by jvm tool and jmx
feature.monitoragent.detector.local.scan.enable=true
#enable container scan by http
Expand Down Expand Up @@ -144,6 +145,8 @@ feature.logagent.MutiThread.enable=true
feature.logagent.MutiThread.thread.max=2
#default filter
feature.logagent.defrule.filter=
#LOGAGENT LOG LEVEL
feature.logagent.log.level=DEBUG

#Proc Detect Agent Feature
feature.procscan.loader=com.creditease.uav.agent.heartbeat.jar
Expand Down
27 changes: 15 additions & 12 deletions com.creditease.uav.agent.buildComponent/config/ma_pro.properties
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ meta.nodetype=\u76D1\u63A7\u4EE3\u7406\u7A0B\u5E8F

#resource components
#resource.xxxx
resource.messageproducer.class=com.creditease.agent.feature.common.MessageProducerResourceComponent
resource.messageproducer.class=org.uavstack.resources.common.messaging.MessageProducerResourceComponent
#resource.xxxx.properties, for example: <key1>=<value1>,<key2>=<value2>,...
resource.messageproducer.properties=
#resource.xxxx.<customized item>
Expand All @@ -27,7 +27,7 @@ resource.messageproducer.messagingthreshold=30
resource.messageproducer.messagesizelimit=262000

#System Http Invoker
resource.httpinvoke.class=com.creditease.agent.feature.common.HttpInvokerResourceComponent
resource.httpinvoke.class=org.uavstack.resources.common.invoker.HttpInvokerResourceComponent
resource.httpinvoke.properties=
resource.httpinvoke.http.client.maxperroute=20
resource.httpinvoke.http.client.maxtotal=100
Expand All @@ -36,14 +36,14 @@ resource.httpinvoke.http.client.conntimeout=10000
resource.httpinvoke.http.client.reqtimeout=10000

#nodeoperagent
feature.nodeoperagent.loader=default
feature.nodeoperagent.loader=com.creditease.uav.agent.heartbeat.jar
#feature.xxx.enable should enable this feature
feature.nodeoperagent.enable=true
feature.nodeoperagent.components=com.creditease.agent.feature.NodeOperAgent
feature.nodeoperagent.http.port=10101

#notifyagent
feature.notifyagent.loader=default
feature.notifyagent.loader=com.creditease.uav.notifymanager.jar
#feature.xxx.enable should enable this feature
feature.notifyagent.enable=true
#feature.xxx.components is the component classes (extends com.creditease.agent.spi.AgentFeatureComponent), separator "," for multiply classes
Expand All @@ -58,7 +58,7 @@ feature.notifyagent.frozenTime=300

#heartbeatclientagent Agent Feature xxx
#feature.xxx.loader should be a jar file path or default
feature.hbclientagent.loader=default
feature.hbclientagent.loader=com.creditease.uav.agent.heartbeat.jar
#feature.xxx.enable should enable this feature
feature.hbclientagent.enable=true
#feature.xxx.components is the component classes (extends com.creditease.agent.spi.AgentFeatureComponent), separator "," for multiply classes
Expand All @@ -73,7 +73,7 @@ feature.hbclientagent.interval=15000
#agent feature configuration
#Monitor Agent Feature xxx
#feature.xxx.loader should be a jar file path or default
feature.monitoragent.loader=default
feature.monitoragent.loader=com.creditease.uav.agent-1.0.jar
#feature.xxx.enable should enable this feature
feature.monitoragent.enable=true
#feature.xxx.components is the component classes (extends com.creditease.agent.spi.AgentFeatureComponent), separator "," for multiply classes
Expand Down Expand Up @@ -101,10 +101,11 @@ feature.monitoragent.http.enable=false
feature.monitoragent.jvm.filters=com.sun.tools,sun.tools,jenkins-cli.jar,com.creditease.agent.feature.nodeopagent.NodeOperCtrlClient
#monitoragent detect interval default is 30000
feature.monitoragent.detector.interval=30000
feature.monitoragent.detector.profilehbtimeout=15000
#enable local scan by jvm tool and jmx
feature.monitoragent.detector.local.scan.enable=true
#enable container scan by http
feature.monitoragent.detector.container.scan.enable=false
feature.monitoragent.detector.container.scan.enable=true
#container ports
feature.monitoragent.detector.container.ports=
#monitoragent detect interval default is 15000
Expand All @@ -118,7 +119,7 @@ feature.monitoragent.limit.cpu.timerange=120000

#Log Agent Feature xxx
#feature.xxx.loader should be a jar file path or default
feature.logagent.loader=default
feature.logagent.loader=com.creditease.uav.agent-1.0.jar
#feature.xxx.enable should enable this feature
feature.logagent.enable=true
#feature.xxx.enable should enable the self Log collection feature
Expand All @@ -136,9 +137,11 @@ feature.logagent.MutiThread.enable=true
feature.logagent.MutiThread.thread.max=2
#default filter
feature.logagent.defrule.filter=
#LOGAGENT LOG LEVEL
feature.logagent.log.level=DEBUG

#Proc Detect Agent Feature
feature.procscan.loader=default
feature.procscan.loader=com.creditease.uav.agent.heartbeat.jar
feature.procscan.enable=true
feature.procscan.components=com.creditease.agent.feature.ProcDetectAgent
feature.procscan.properties=
Expand All @@ -148,13 +151,13 @@ feature.procscan.networkDetect.interval=15000
feature.procscan.networkDetect.collectTime=2000

# Process Watch Feature
feature.procwatch.loader=default
feature.procwatch.loader=com.creditease.uav.agent.heartbeat.jar
feature.procwatch.enable=true
feature.procwatch.interval=15000
feature.procwatch.components=com.creditease.agent.feature.ProcWatchAgent

# Data Collect Feature
feature.collectclient.loader=default
feature.collectclient.loader=com.creditease.uav.collect.jar
feature.collectclient.enable=true
feature.collectclient.components=com.creditease.uav.collect.client.CollectDataAgent
feature.collectclient.properties=
Expand All @@ -167,6 +170,6 @@ feature.collectclient.writeposinterval=3000
feature.collectclient.idlecheckerdelay=12000
feature.collectclient.idlecheckerinterval=5000
# Thread Analysis Feature
feature.threadanalysis.loader=default
feature.threadanalysis.loader=com.creditease.uav.threadanalysis.jar
feature.threadanalysis.enable=true
feature.threadanalysis.components=com.creditease.uav.threadanalysis.client.ThreadAnalysisAgent
Loading

0 comments on commit 0874d5f

Please sign in to comment.