Hadoop @ LinkedIn
LinkedIn relies heavily on Hadoop for our offline data infrastructure needs (see our many previous blog posts on the subject), and we …
Open-Sourcing the LinkedIn Gradle Plugin and DSL for Apache Hadoop
We developed the Hadoop Plugin to help our Hadoop application developers more effectively build, test and deploy Hadoop …
White Elephant: The Hadoop Tool You Never Knew You Needed
As the use of Hadoop grows in an organization, scheduling, capacity planning, and billing become critical concerns. These are all …
Dali New - LinkedIn
This is especially true at a company like LinkedIn and the data they process must evolve rapidly over time. Dali is a collection of …
Rewinder: Interactive Analysis of Hadoop's Computational Resources
In order to gain insights from the massive amounts of structured and unstructured data at LinkedIn, we leverage the Hadoop …
Recap: Improving Hadoop Performance by (up to) 1000x
At LinkedIn, we not only use Hadoop extensively, but we also contribute significantly to its development, as well as to related open …
Dr. Elephant - LinkedIn
A performance monitoring and tuning tool for Hadoop and Spark, Dr. Elephant automatically gathers all the metrics, runs analysis on …
User Engagement Powered By Apache Pig and Hadoop
This blog post tells the story of how you can use Apache Pig and Hadoop to turn terabytes of data into user engagement. It starts …
Search | LinkedIn Engineering
I lead the Data Analytics Infrastructure team at LinkedIn; we work on core infrastructure components such as Hadoop and Spark, as …
Streams & Stream Processing
Samza Apache Samza is a distributed stream processing framework. It uses Apache Kafka for messaging, and Apache Hadoop …