If you are looking for a solution for Big Data in the Cloud, Qubole looks very interesting. Qubole was founded by Ashish Thusoo and Joydeep Sen Sarma who managed Facebook’s Big Data team and were the developers of what is today Apache Hive. They also have Shrikanth Shankar on the team who was the architect of Oracle’s Exalytics platform. Some very serious players indeed – probably one of the most experienced Big Data teams in the world. They have started their own company, Qubole, to deliver a Big Data Service – basically Big Data in the Cloud. They have a number of solid reference customers including Pinterest. You can find out more about Qubole’s Big Data in the Cloud here.
A lot of things around Big Data seem to be very much a “roll your own” approach. Since there is a lot of open source and lot of fragmentation, you spend a lot of time figuring out what parts you need and then how to put things together. Then you have to figure out how to get your data into what you have built in an efficient manner and then how to get it out for analytics. Companies like Teradata and Oracle have appliances that make this easier but they are expensive and when you are first figuring things out, you may not want to spend that much money to start with. Qubole gives you an easy way to start out because everything is in the cloud and you can start out cheaply and then scale to what you need. Plus, people like Ashish, Joydeep, and Shrikanth were doing this stuff before the term “Big Data” became so popular. They were extremely successful with what they did at Facebook and Oracle and now you can leverage their experience and know how at a very economical price.