View Praveen Gandluri's profile on LinkedIn
Home Posts

Solution Blueprint for Big Data on the Cloud Proof-Of-Concept


Nowadays the number one priority in CIOs' agenda is Big Data and Cloud adoption in the organizations. Many companies have started with Big Data Proof-Of-Concepts (POC), and you would hear a lot of lessons learnt in any conference you attend these days. Though there are many success stories, some POC attempts don't quite end up as expected and companies start to rethink their need for Big Data.
Two ways in which POCs fail are,This document provides you with a Solution blueprint for the holistic Big Data on the Cloud Proof-Of-Concept, a straight forward and simple approach to start with, yet robust enough to easily productionalize without much modifications.
As part of the Solution blueprint, various tools and features available in Amazon Web Services (AWS) are used. Below is the list of the AWS Services used:This document provides detailed steps, readily usable code snippets and screenshots where necessary so that, with very slight modifications (such as updating your AWS credentials and input path etc.,) you can have your POC done.
The main objective is to reduce the time to finish the POC (not months or weeks but in days or even less) and make it easy to Productionalize when ready.

Below is the architectural diagram:

aws emr s3 redshift data pipeline

Technical and process details of this demonstration: Let's get started!!

Disclaimer: This is a personal blog. Any views or opinions represented in this blog are personal and belong solely to the author and do not necesserily represent the author's employer or the clients the author works for. All content provided on this blog is for informational purposes only. The author will not be liable for any errors or omissions in this information nor for the availability of this information. All trademarks, logos,icons and images cited herein are the property of their respective owners.