Skip to content

Commit

Permalink
v1.6.0 development (#66)
Browse files Browse the repository at this point in the history 
* fix issue #62, mapping of Zeek values to Moloch's http.uri

* fix issue #63, "View JSON Document" context action for ID field in Moloch does not display old data correctly bug moloch

* bump version for development of 1.5.3

* a better way to handle issue #60, create the right-click actions in the WISE source rather than config.ini

* add Zeek support for GQUIC (issue #64)

* Added QUIC dashboards (issue #64)

* change version for release from 1.5.3 to 1.6.0
  • Loading branch information
@mmguero
mmguero authored Sep 30, 2019
1 parent d453713 commit d1479a1
Show file tree
Hide file tree
Showing 57 changed files with 477 additions and 116 deletions.
9 changes: 9 additions & 0 deletions Dockerfiles/moloch.Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,15 @@ RUN sed -i "s/buster main/buster main contrib non-free/g" /etc/apt/sources.list
./configure --bro-dist="/data/bro-"$ZEEK_VERSION --install-root=$ZEEK_DIR/lib/bro/plugins && \
make && \
make install && \
git clone --depth 1 https://github.com/salesforce/GQUIC_Protocol_Analyzer /tmp/gquic && \
cd /data/bro-$ZEEK_VERSION/aux/bro-aux/plugin-support/ && \
./init-plugin ./bro-quic Salesforce GQUIC && \
cd ./bro-quic && \
rm -rf CMakeLists.txt ./scripts ./src && \
cp -vr /tmp/gquic/CMakeLists.txt /tmp/gquic/scripts /tmp/gquic/src ./ && \
./configure --bro-dist="/data/bro-"$ZEEK_VERSION --install-root=$ZEEK_DIR/lib/bro/plugins && \
make && \
make install && \
cd $MOLOCHDIR/doc/images && \
find . -name "*.png" -exec bash -c 'convert "{}" -fuzz 2% -transparent white -background white -alpha remove -strip -interlace Plane -quality 85% "{}.jpg" && rename "s/\.png//" "{}.jpg"' \; && \
cd $MOLOCHDIR/doc && \
Expand Down
44 changes: 22 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,17 +112,17 @@ You can then observe that the images have been retrieved by running `docker imag
```
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
malcolmnetsec/moloch 1.5.2 xxxxxxxxxxxx 27 minutes ago 517MB
malcolmnetsec/htadmin 1.5.2 xxxxxxxxxxxx 2 hours ago 180MB
malcolmnetsec/nginx-proxy 1.5.2 xxxxxxxxxxxx 4 hours ago 53MB
malcolmnetsec/file-upload 1.5.2 xxxxxxxxxxxx 24 hours ago 198MB
malcolmnetsec/pcap-capture 1.5.2 xxxxxxxxxxxx 24 hours ago 111MB
malcolmnetsec/file-monitor 1.5.2 xxxxxxxxxxxx 24 hours ago 355MB
malcolmnetsec/logstash-oss 1.5.2 xxxxxxxxxxxx 25 hours ago 1.24GB
malcolmnetsec/curator 1.5.2 xxxxxxxxxxxx 25 hours ago 303MB
malcolmnetsec/kibana-oss 1.5.2 xxxxxxxxxxxx 33 hours ago 944MB
malcolmnetsec/filebeat-oss 1.5.2 xxxxxxxxxxxx 11 days ago 459MB
malcolmnetsec/elastalert 1.5.2 xxxxxxxxxxxx 11 days ago 276MB
malcolmnetsec/moloch 1.6.0 xxxxxxxxxxxx 27 minutes ago 517MB
malcolmnetsec/htadmin 1.6.0 xxxxxxxxxxxx 2 hours ago 180MB
malcolmnetsec/nginx-proxy 1.6.0 xxxxxxxxxxxx 4 hours ago 53MB
malcolmnetsec/file-upload 1.6.0 xxxxxxxxxxxx 24 hours ago 198MB
malcolmnetsec/pcap-capture 1.6.0 xxxxxxxxxxxx 24 hours ago 111MB
malcolmnetsec/file-monitor 1.6.0 xxxxxxxxxxxx 24 hours ago 355MB
malcolmnetsec/logstash-oss 1.6.0 xxxxxxxxxxxx 25 hours ago 1.24GB
malcolmnetsec/curator 1.6.0 xxxxxxxxxxxx 25 hours ago 303MB
malcolmnetsec/kibana-oss 1.6.0 xxxxxxxxxxxx 33 hours ago 944MB
malcolmnetsec/filebeat-oss 1.6.0 xxxxxxxxxxxx 11 days ago 459MB
malcolmnetsec/elastalert 1.6.0 xxxxxxxxxxxx 11 days ago 276MB
docker.elastic.co/elasticsearch/elasticsearch-oss 6.8.3 xxxxxxxxxxxx 5 weeks ago 769MB
```

Expand Down Expand Up @@ -1360,17 +1360,17 @@ Pulling nginx-proxy ... done
user@host:~/Malcolm$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
malcolmnetsec/moloch 1.5.2 xxxxxxxxxxxx 27 minutes ago 517MB
malcolmnetsec/htadmin 1.5.2 xxxxxxxxxxxx 2 hours ago 180MB
malcolmnetsec/nginx-proxy 1.5.2 xxxxxxxxxxxx 4 hours ago 53MB
malcolmnetsec/file-upload 1.5.2 xxxxxxxxxxxx 24 hours ago 198MB
malcolmnetsec/pcap-capture 1.5.2 xxxxxxxxxxxx 24 hours ago 111MB
malcolmnetsec/file-monitor 1.5.2 xxxxxxxxxxxx 24 hours ago 355MB
malcolmnetsec/logstash-oss 1.5.2 xxxxxxxxxxxx 25 hours ago 1.24GB
malcolmnetsec/curator 1.5.2 xxxxxxxxxxxx 25 hours ago 303MB
malcolmnetsec/kibana-oss 1.5.2 xxxxxxxxxxxx 33 hours ago 944MB
malcolmnetsec/filebeat-oss 1.5.2 xxxxxxxxxxxx 11 days ago 459MB
malcolmnetsec/elastalert 1.5.2 xxxxxxxxxxxx 11 days ago 276MB
malcolmnetsec/moloch 1.6.0 xxxxxxxxxxxx 27 minutes ago 517MB
malcolmnetsec/htadmin 1.6.0 xxxxxxxxxxxx 2 hours ago 180MB
malcolmnetsec/nginx-proxy 1.6.0 xxxxxxxxxxxx 4 hours ago 53MB
malcolmnetsec/file-upload 1.6.0 xxxxxxxxxxxx 24 hours ago 198MB
malcolmnetsec/pcap-capture 1.6.0 xxxxxxxxxxxx 24 hours ago 111MB
malcolmnetsec/file-monitor 1.6.0 xxxxxxxxxxxx 24 hours ago 355MB
malcolmnetsec/logstash-oss 1.6.0 xxxxxxxxxxxx 25 hours ago 1.24GB
malcolmnetsec/curator 1.6.0 xxxxxxxxxxxx 25 hours ago 303MB
malcolmnetsec/kibana-oss 1.6.0 xxxxxxxxxxxx 33 hours ago 944MB
malcolmnetsec/filebeat-oss 1.6.0 xxxxxxxxxxxx 11 days ago 459MB
malcolmnetsec/elastalert 1.6.0 xxxxxxxxxxxx 11 days ago 276MB
docker.elastic.co/elasticsearch/elasticsearch-oss 6.8.3 xxxxxxxxxxxx 5 weeks ago 769MB
```

Expand Down
22 changes: 11 additions & 11 deletions docker-compose-standalone-zeek-live.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ services:
- ./elasticsearch:/usr/share/elasticsearch/data:delegated
- ./elasticsearch-backup:/opt/elasticsearch/backup:delegated
kibana:
image: malcolmnetsec/kibana-oss:1.5.2
image: malcolmnetsec/kibana-oss:1.6.0
restart: "no"
hostname: kibana
environment:
Expand All @@ -126,7 +126,7 @@ services:
retries: 3
start_period: 200s
elastalert:
image: malcolmnetsec/elastalert:1.5.2
image: malcolmnetsec/elastalert:1.6.0
restart: "no"
hostname: elastalert
environment:
Expand All @@ -151,7 +151,7 @@ services:
- ./elastalert/config/config.json:/opt/elastalert-server/config/config.json
- ./elastalert/rules/:/opt/elastalert/rules/
curator:
image: malcolmnetsec/curator:1.5.2
image: malcolmnetsec/curator:1.6.0
restart: "no"
hostname: curator
environment:
Expand All @@ -161,7 +161,7 @@ services:
depends_on:
- elasticsearch
logstash:
image: malcolmnetsec/logstash-oss:1.5.2
image: malcolmnetsec/logstash-oss:1.6.0
restart: "no"
hostname: logstash
environment:
Expand Down Expand Up @@ -189,7 +189,7 @@ services:
- ./cidr-map.txt:/usr/share/logstash/config/cidr-map.txt:ro
- ./host-map.txt:/usr/share/logstash/config/host-map.txt:ro
filebeat:
image: malcolmnetsec/filebeat-oss:1.5.2
image: malcolmnetsec/filebeat-oss:1.6.0
restart: "no"
hostname: filebeat
environment:
Expand All @@ -214,7 +214,7 @@ services:
- ./filebeat/certs/client.crt:/certs/client.crt:ro
- ./filebeat/certs/client.key:/certs/client.key:ro
moloch:
image: malcolmnetsec/moloch:1.5.2
image: malcolmnetsec/moloch:1.6.0
restart: "no"
hostname: moloch
env_file:
Expand Down Expand Up @@ -246,7 +246,7 @@ services:
- ./moloch-logs:/data/moloch/logs
- ./moloch-raw:/data/moloch/raw
file-monitor:
image: malcolmnetsec/file-monitor:1.5.2
image: malcolmnetsec/file-monitor:1.6.0
restart: "no"
hostname: filemon
environment:
Expand All @@ -257,7 +257,7 @@ services:
- ./zeek-logs/extract_files:/data/zeek/extract_files
- ./zeek-logs/current:/data/zeek/logs
pcap-capture:
image: malcolmnetsec/pcap-capture:1.5.2
image: malcolmnetsec/pcap-capture:1.6.0
restart: "no"
network_mode: host
ulimits:
Expand All @@ -274,7 +274,7 @@ services:
volumes:
- ./pcap/upload:/pcap
upload:
image: malcolmnetsec/file-upload:1.5.2
image: malcolmnetsec/file-upload:1.6.0
restart: "no"
hostname: upload
env_file:
Expand All @@ -291,7 +291,7 @@ services:
volumes:
- ./pcap/upload:/var/www/upload/server/php/chroot/files
htadmin:
image: malcolmnetsec/htadmin:1.5.2
image: malcolmnetsec/htadmin:1.6.0
restart: "no"
hostname: htadmin
environment:
Expand All @@ -303,7 +303,7 @@ services:
- ./htadmin/metadata:/var/www/htadmin/config/metadata:rw
- ./nginx/htpasswd:/var/www/htadmin/config/htpasswd:rw
nginx-proxy:
image: malcolmnetsec/nginx-proxy:1.5.2
image: malcolmnetsec/nginx-proxy:1.6.0
restart: "no"
hostname: nginx-proxy
depends_on:
Expand Down
22 changes: 11 additions & 11 deletions docker-compose-standalone.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ services:
- ./elasticsearch:/usr/share/elasticsearch/data:delegated
- ./elasticsearch-backup:/opt/elasticsearch/backup:delegated
kibana:
image: malcolmnetsec/kibana-oss:1.5.2
image: malcolmnetsec/kibana-oss:1.6.0
restart: "no"
hostname: kibana
environment:
Expand All @@ -126,7 +126,7 @@ services:
retries: 3
start_period: 200s
elastalert:
image: malcolmnetsec/elastalert:1.5.2
image: malcolmnetsec/elastalert:1.6.0
restart: "no"
hostname: elastalert
environment:
Expand All @@ -151,7 +151,7 @@ services:
- ./elastalert/config/config.json:/opt/elastalert-server/config/config.json
- ./elastalert/rules/:/opt/elastalert/rules/
curator:
image: malcolmnetsec/curator:1.5.2
image: malcolmnetsec/curator:1.6.0
restart: "no"
hostname: curator
environment:
Expand All @@ -161,7 +161,7 @@ services:
depends_on:
- elasticsearch
logstash:
image: malcolmnetsec/logstash-oss:1.5.2
image: malcolmnetsec/logstash-oss:1.6.0
restart: "no"
hostname: logstash
environment:
Expand Down Expand Up @@ -189,7 +189,7 @@ services:
- ./cidr-map.txt:/usr/share/logstash/config/cidr-map.txt:ro
- ./host-map.txt:/usr/share/logstash/config/host-map.txt:ro
filebeat:
image: malcolmnetsec/filebeat-oss:1.5.2
image: malcolmnetsec/filebeat-oss:1.6.0
restart: "no"
hostname: filebeat
environment:
Expand All @@ -214,7 +214,7 @@ services:
- ./filebeat/certs/client.crt:/certs/client.crt:ro
- ./filebeat/certs/client.key:/certs/client.key:ro
moloch:
image: malcolmnetsec/moloch:1.5.2
image: malcolmnetsec/moloch:1.6.0
restart: "no"
hostname: moloch
env_file:
Expand Down Expand Up @@ -246,7 +246,7 @@ services:
- ./moloch-logs:/data/moloch/logs
- ./moloch-raw:/data/moloch/raw
file-monitor:
image: malcolmnetsec/file-monitor:1.5.2
image: malcolmnetsec/file-monitor:1.6.0
restart: "no"
hostname: filemon
environment:
Expand All @@ -257,7 +257,7 @@ services:
- ./zeek-logs/extract_files:/data/zeek/extract_files
- ./zeek-logs/current:/data/zeek/logs
pcap-capture:
image: malcolmnetsec/pcap-capture:1.5.2
image: malcolmnetsec/pcap-capture:1.6.0
restart: "no"
network_mode: host
ulimits:
Expand All @@ -274,7 +274,7 @@ services:
volumes:
- ./pcap/upload:/pcap
upload:
image: malcolmnetsec/file-upload:1.5.2
image: malcolmnetsec/file-upload:1.6.0
restart: "no"
hostname: upload
env_file:
Expand All @@ -291,7 +291,7 @@ services:
volumes:
- ./pcap/upload:/var/www/upload/server/php/chroot/files
htadmin:
image: malcolmnetsec/htadmin:1.5.2
image: malcolmnetsec/htadmin:1.6.0
restart: "no"
hostname: htadmin
environment:
Expand All @@ -303,7 +303,7 @@ services:
- ./htadmin/metadata:/var/www/htadmin/config/metadata:rw
- ./nginx/htpasswd:/var/www/htadmin/config/htpasswd:rw
nginx-proxy:
image: malcolmnetsec/nginx-proxy:1.5.2
image: malcolmnetsec/nginx-proxy:1.6.0
restart: "no"
hostname: nginx-proxy
depends_on:
Expand Down
22 changes: 11 additions & 11 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/kibana.Dockerfile
image: malcolmnetsec/kibana-oss:1.5.2
image: malcolmnetsec/kibana-oss:1.6.0
restart: "no"
hostname: kibana
environment:
Expand All @@ -132,7 +132,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/elastalert.Dockerfile
image: malcolmnetsec/elastalert:1.5.2
image: malcolmnetsec/elastalert:1.6.0
restart: "no"
hostname: elastalert
environment:
Expand Down Expand Up @@ -160,7 +160,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/curator.Dockerfile
image: malcolmnetsec/curator:1.5.2
image: malcolmnetsec/curator:1.6.0
restart: "no"
hostname: curator
environment:
Expand All @@ -175,7 +175,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/logstash.Dockerfile
image: malcolmnetsec/logstash-oss:1.5.2
image: malcolmnetsec/logstash-oss:1.6.0
restart: "no"
hostname: logstash
environment:
Expand Down Expand Up @@ -213,7 +213,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/filebeat.Dockerfile
image: malcolmnetsec/filebeat-oss:1.5.2
image: malcolmnetsec/filebeat-oss:1.6.0
restart: "no"
hostname: filebeat
environment:
Expand Down Expand Up @@ -242,7 +242,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/moloch.Dockerfile
image: malcolmnetsec/moloch:1.5.2
image: malcolmnetsec/moloch:1.6.0
restart: "no"
hostname: moloch
env_file:
Expand Down Expand Up @@ -280,7 +280,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/file-monitor.Dockerfile
image: malcolmnetsec/file-monitor:1.5.2
image: malcolmnetsec/file-monitor:1.6.0
restart: "no"
hostname: filemon
environment:
Expand All @@ -294,7 +294,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/pcap-capture.Dockerfile
image: malcolmnetsec/pcap-capture:1.5.2
image: malcolmnetsec/pcap-capture:1.6.0
restart: "no"
network_mode: host
ulimits:
Expand All @@ -314,7 +314,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/file-upload.Dockerfile
image: malcolmnetsec/file-upload:1.5.2
image: malcolmnetsec/file-upload:1.6.0
restart: "no"
hostname: upload
env_file:
Expand All @@ -331,7 +331,7 @@ services:
volumes:
- ./pcap/upload:/var/www/upload/server/php/chroot/files
htadmin:
image: malcolmnetsec/htadmin:1.5.2
image: malcolmnetsec/htadmin:1.6.0
build:
context: .
dockerfile: Dockerfiles/htadmin.Dockerfile
Expand All @@ -349,7 +349,7 @@ services:
build:
context: .
dockerfile: Dockerfiles/nginx.Dockerfile
image: malcolmnetsec/nginx-proxy:1.5.2
image: malcolmnetsec/nginx-proxy:1.6.0
restart: "no"
hostname: nginx-proxy
depends_on:
Expand Down
Loading

0 comments on commit d1479a1

Please sign in to comment.