The following diagram shows a typical Big Data Infrastructure Design. This is from one of Allied Consultant’s Big Data works.
Cluster Expected Volume Benchmark hardware Project Hardware requirements Cores RAM # nodes Disk Source 6 Million records / month ~ 3 records per second HDFS 6 million/month 1 namenode, 20 datanodes, 2 CPU/node, 64GB RAM/node 1 6G 1 : Master 3: Slaves 120% of 6G =7.2GB/month Kafka 4 topics 6 million/month per topic 1 nodes […]
With the help of Big data, Small Businesses can gain the competitive edge they require to stay ahead of the curve. For beginners, Big data includes large data sets of information that can reveal insights about your customers to help you make valuable business decisions. Data can help you to compose an overall strategy for […]
Data Scientists are known for having a knack for statistics, data analysis, etc. to understand and obtain insights from a given dataset, usually quite enormous in quantity. If you don’t think you have this skill set, but the career appeals to you, you can always take a course like a springboard data science which will […]
What is a Dashboard? A Dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance. A user interface that organizes, integrates, and presents mission-critical information, pulled from multiple sources to users in […]
I have been involved in consulting for about 20 years now, and I think one of the biggest challenges (in the long term) for consulting firms is building a sense of ownership and fairness within the company. One of the biggest problems with people, in general, is that they overvalue what they do and undervalue […]
Resource Management in Information Technology There is a whole host of technology available nowadays to ensure that your IT hardware resources manages efficiently. You may have a data center in-house, and a few cloud nodes/services/apps which together may constitute your investment in hardware. That would translate to resource capacity: memory, disk, and processor. In the […]
1- Download Docker for windows. 2- Install Docker for windows.exe 3- Check whether docker is running 4- View/update docker settings 5- Now Lets start using docker…. – docker #displays docker help – docker version #check docker version – docker images #check all images – docker ps #check current running containers – docker ps -a #check […]
This is a brief guide on a Multi Node Kafka Cluster Setup using Windows Docker: —————————————– Setting Up a Three Node Kafka Cluster —————————————– 1- Start ZooKeeper and Kafka using Docker Compose up command docker-compose up 2- In another terminal window, go to the same directory (kafka-cluster). Before we move on, let’s make sure the services […]
Introducing K2 as a Process Automation Platform HR consists of a wide variety of processes with integrated levels of checks & verifications across departments. K2 helps automate these HR processes through workflows that can increase process visibility, eliminate redundant operations and reduce repetitive, manual labor errors. K2’s HR management software solutions enable developers to solve […]