A Comparison of different Big Data Processing Platforms

Cluster	Expected Volume	Benchmark hardware	Project Hardware requirements
			Cores	RAM	# nodes	Disk
Source	6 Million records / month ~ 3 records per second
HDFS	6 million/month	1 namenode, 20 datanodes, 2 CPU/node, 64GB RAM/node	1	6G	1 : Master 3: Slaves	120% of 6G =7.2GB/month
Kafka	4 topics 6 million/month per topic	1 nodes @ 4 GB RAM, 1 CPU,200 GB disk each 16,000 Msg/sec	1	4G	1	24 GB/month
Kafka Connector for HBASE		N/A
Kafka Connector for HDFS		N/A
Logstash	6 million/month	1 node, 3.75GB RAM, 1 CPU Cores 180 Events/sec	1	4G	1
Hbase	6GB/month	7 nodes, 32gb RAM, 8 CPU cores 60240 req/sec – 200 req/sec	1	8G	1	4GB/month