At Obstkel, we focus on helping our clients make better business decisions by leveraging their data, and our technical expertise in everything Data.
In this post we will address Spark SQL Functions, i.e., built-in functions, its syntax and what it does. The comprehensive list of Spark functions provided in the Apache Spark API documentation can be a bit difficult to navigate. In this post we breakdown the Apache Spark built-in functions by Category: Operators, String functions, Number functions, Date functions, Array functions, Conversion functions and Regex functions.
Read MoreIn this post we will address Spark SQL Date Functions, its syntax and what it does. There are 28 Spark SQL Date functions, meant to address string to date, date to timestamp, timestamp to date, date additions, subtractions and current date conversions.
Read MoreThis post focuses on answering some of the frequently asked Sqoop interview questions. Apache Sqoop is a relatively small tool in the Apache Hadoop ecosystem, so most technical interviews do not contain any more than a question or two on Sqoop. The list of interview questions provided below are a consolidation of our top 9 FAQ.
Read MoreIf your are confused about what PySpark is, then fret no more. This article is focused on answering some of the common questions such as "What is PySpark", "Why learn PySpark", "What is the difference between the PySpark ML packages" and "What does PySpark contain".
Read MoreThis post elaborates on Apache Spark transformation and action operations by providing a step by step walk through of spark scala examples. These examples are in no particular sequence and is the first part of our five part spark scala examples post.
Read MoreIf your are a beginner, understanding the basics of Apache Spark will help you build a strong foundation before you get to the more complex concepts. Often times these concepts are intertwined with new terminology. Associating the terminology with each other in a hierarchical manner to build relationships has been determined to help assimilate information effectively, and in a shorter time frame.
Read MoreComparable to the amount of interest around Big Data and Apache Hadoop, is the confusion around the terminology. Things get even murkier when we delve into the various projects within Apache Hadoop. What is Apache Spark ? How does it relate to Cloudera Spark ? In this post, we will address some of the frequent questions around how we define Spark, Cloudera Spark and Apache spark
Read MoreLearning a new skill is challenging; to ensure success, its important to create an optimal environment with least resistance. One of the first steps to getting hands on with Hadoop is downloading, installing and configuring the Hadoop ecosystem and its application components. With all the different components, downloading & installing Apache Hadoop can become a nightmare.
Read MoreOf the numerous Apache Hadoop File System (HDFS) commands, there are a handful that get used on a frequent basis. Having a solid grasp of these 8 basic hdfs commands will get you through most of your daily grind as a developer. This top 7 list is intended for beginners and is intentionally created as a "skinny" version without all the options
Read More