danielguerra/bro-debian-elasticsearch
Bro on debian with full elasticsearch-2 kibana intergration.
10K+
danielguerra/bro-debian-elasticsearch repository overview
BRO ELK AMQP docker integration
| Bro IDS | Elasticsearch + Kibana | RabbitMQ |
|---|---|---|
 |  |  |
| 2.4.1 | 2+ / 4+ | 3.5.7 |
About
Integrates Bro IDS git with Elasticsearch 2+ & Kibana 4+ Bro was compiled with broker,rocksdb and pybroker (full featured).Bro can write directly into Elasticsearch without logstash. The bro scripts have been modified in order to satisfy elasticsearch. The example below uses 3 elasticsearch nodes. The container bro-xinetd writes to the master. Kibana reads from node02. The commandline bro uses node01. Added amqp (rabbitmq) consume/publish roles with the debian amqp-tools. Todo elasticsearch amqp consumer, amqpfs for extracted files.
Dependencies
debian:jessie
elasticsearch 2.1
kibana 4.3
rabbitmq 3.5.6-management
Image Size
Docker-compose
The simplest way to start all nodes is using docker-compose
wget https://raw.githubusercontent.com/danielguerra69/bro-debian-elasticsearch/master/docker-compose.yml
export DOCKERHOST=<dockerhost-ip>
export COMPOSE_API_VERSION=1.18
docker-compose pull
docker-compose up
This compose file starts a role/xinetd-forensic which currently supports pcap and extracted file access from kibana. It listens to port 1969 for pcap files.
nc <dockerhost-ip> 1969 < my.pcap
tcpdump -i eth0 -s 0 -w - not host <dockerhost-ip> | nc <dockerhost-ip> 1969
Kibana is viewed in your browser. http://:5601/
The pcap and extracted data can be reached over tcp port 80
Developers
Full version with all tools and sources to build this project. Sources are in /tmp.
docker pull danielguerra/bro-debian-elasticsearch:develop
Installation
Before you begin I recommend to start with pulling fresh images.
docker pull danielguerra/bro-debian-elasticsearch
docker pull elasticsearch:2.1 (or latest)
docker pull kibana:4.3 (or latest)
docker pull rabbitmq:3.5.6-management
elastic data
Create empty elasticsearch data volumes optional,if not remove --volumes-from ...
docker create -v /usr/share/elasticsearch/data --name elastic-data-master danielguerra/empty-elastic-data /bin/true
docker create -v /usr/share/elasticsearch/data --name elastic-data-node01 danielguerra/empty-elastic-data /bin/true
docker create -v /usr/share/elasticsearch/data --name elastic-data-node02 danielguerra/empty-elastic-data /bin/true
elasticsearch
Run three elasticsearch nodes (minimal)
docker run -d --volumes-from elastic-data-master --hostname=elasticsearch-master --name elasticsearch-master elasticsearch -Des.network.bind_host=elasticsearch-master --cluster.name=bro --node.name=elasticsearch-master --discovery.zen.ping.multicast.enabled=false --network.host=elasticsearch-master
docker run -d --volumes-from elastic-data-node01 --hostname=elasticsearch-node01 --name elasticsearch-node01 --link=elasticsearch-master:master elasticsearch -Des.network.bind_host=elasticsearch-node01 --cluster.name=bro --node.name=elasticsearch-node01 --discovery.zen.ping.unicast.hosts=master:9300 --network.host=elasticsearch-node01
docker run -d --volumes-from elastic-data-node02 --hostname=elasticsearch-node02 --name elasticsearch-node02 --link=elasticsearch-master:master elasticsearch -Des.network.bind_host=elasticsearch-node02 --cluster.name=bro --node.name=elasticsearch-node02 --discovery.zen.ping.unicast.hosts=master:9300 --network.host=elasticsearch-node02
elasticsearch mapping (important)
After you have a running elasticsearch-cluster you should start a commandline bro and do
docker run --link elasticsearch-master:elasticsearch --rm danielguerra/bro-debian-elasticsearch /scripts/bro-mapping.sh
kibana
Configure kibana
docker run --rm --link elasticsearch-master:elasticsearch danielguerra/bro-kibana-config
Start kibana
docker run -d -p 5601:5601 --link=elasticsearch-node02:elasticsearch --hostname=kibana --name kibana kibana
Point your browser http://:5601
bro on the commandline
commandline and local file log
docker run -ti -v /Users/PCAP:/pcap --name bro-log danielguerra/bro-debian-elasticsearch
commandline and log to elasticsearch
docker run -ti --link elasticsearch-node01:elasticsearch -v /Users/PCAP:/pcap --name bro danielguerra/bro-debian-elasticsearch /role/cmd-elasticsearch
readfiles from bro commandline
bro -r /pcap/mydump.pcap
bro develop version (all sources are in /tmp)
docker run -ti --link elasticsearch-node01:elasticsearch -v /Users/PCAP:/pcap --name bro danielguerra/bro-debian-elasticsearch:develop /role/cmd-elasticsearch
bro xinetd service
when role/xinetd is used no local logs are written, all logs go to elasticsearch
docker run -d -p 1969:1969 --link elasticsearch-master:elasticsearch --name bro-xinetd --hostname bro-xinetd danielguerra/bro-debian-elasticsearch /role/xinetd-elasticsearch
tcpdump to your container from a remote host, replace dockerhost with your ip
tcpdump -i eth0 -s 0 -w /dev/stdout | nc dockerhost 1969
or read a file file to your container
nc dockerhost 1969 < mydump.pcap
bro xinetd forensic
when role/xinetd-forensic is used, pcap and extracted files are available from kibana.
docker run -d -p 1969:1969 -p 80:80 --link elasticsearch-master:elasticsearch --name bro-xinetd-forensic --hostname bro-xinetd-forensic danielguerra/bro-debian-elasticsearch /role/xinetd-forensic
bro ssh server
for bro nodes or just remote key based authentication create an empty ssh volume
docker create -v /root/.ssh --name ssh-container danielguerra/ssh-container /bin/true
create your own keys on your own machine
docker run --volumes-from ssh-container debian:jessie ssh-keygen -q
add your pub key to authorized_keys file
docker run --volumes-from ssh-container debian:jessie cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
create a copy in your directory (pwd)
docker run --volumes-from ssh-container -v $(pwd):/backup debian:jessie cp -R /root/.ssh/* /backup
start bro as ssh daemon
docker run -d -p 1922:22 --link elasticsearch:elasticsearch --name bro-dev danielguerra/bro-debian-elasticsearch /role/sshd
ssh -p 1922 -i id_rsa root@dockerhost
bro amqp
Bro can be used with amqp in elasticsearch out or amqp output
First we need an amqp, this case a rabbitmq
docker run -d -p 8080:15672 --name=rabbitmq --hostname=rabbitmq rabbitmq:3.5.6-management
docker inspect rabbitmq (to get the ip)
Now we can start a bro xinetd service which outputs to rabbitmq
docker run -d -p 1970:1969 --name bro-xinetd-amqp --hostname bro-xinetd-amqp danielguerra/bro-debian-elasticsearch /role/xinetd-amqp
Or a bro that reads pcap files from amqp and outputs to amqp
docker run -d --name=bro-amqp-amqp --hostname=bro-amqp-amqp danielguerra/bro-debian-elasticsearch /role/amqp-amqp <user> <pass> <ip> <queue> <user> <pass> <ip> <exchange>
And publish a pcap file from bro-dev commandline
cat <pcap-file> | amqp-publish --url=amqp://<user>:<pass>@<amqp-ip> --exchange=<exchange>
tcpdump containers
Start a bro-xinetd, do a (replace with your container name and with the bro xinetd ip)
docker run --rm --net=container:<container-to-dump> crccheck/tcpdump -i eth0 -w - | nc <bro-xinetd-ip> 1969 &
docker run --rm --net=container:<container-to-dump> danielguerra/bro-debian-elasticsearch:develop /role/dump-elasticsearch
useful scripts
elastic-indices.sh shows elasticsearch indices bro-mapping.sh bro mapping for kibana including geo_point mapping remove-mapping.sh remove the mapping clean-elastic.sh clean elasticsearch from bro data update-intel.sh update intel for bro
Tag summary
Content type
Image
Digest
sha256:40ccfcabd…
Size
510.8 MB
Last updated
over 9 years ago
docker pull danielguerra/bro-debian-elasticsearch