This site uses cookies to deliver our services. By using this site, you acknowledge that you have read and understand our Cookie and Privacy policy. Your use of Kontext website is subject to this policy. Allow Cookies and Dismiss

Spark

Blog posts about Spark

spark scala parquet

Write and Read Parquet Files in Spark/Scala

608 views   2 comments last modified about 5 months ago

In this page, I’m going to demonstrate how to write and read parquet files in Spark/Scala by using Spark SQLContext class. Reference What is parquet format? Go the following project site to understand more about parquet. ...

View detail
lite-log

Install Big Data Tools (Spark, Zeppelin, Hadoop) in Windows for Learning and Practice

296 views   2 comments last modified about 3 months ago

Are you a Windows/.NET developer and willing to learn big data concepts and tools in your Windows? If yes, you can follow the links below to install them in your PC. The installations are usually easier to do in Linux/UNIX but they are not difficult to implement in Windows either since the...

View detail
sql server spark hdfs parquet sqoop

Load Data into HDFS from SQL Server via Sqoop

190 views   0 comments last modified about 3 months ago

This page shows how to import data from SQL Server into Hadoop via Apache Sqoop. Prerequisites Please follow the link below to install Sqoop in your machine if you don’t have one environment ready. ...

View detail
lite-log spark hdfs scala parquet

Write and Read Parquet Files in HDFS through Spark/Scala

656 views   0 comments last modified about 5 months ago

In my previous post, I demonstrated how to write and read parquet files in Spark/Scala. The parquet file destination is a local folder. Write and Read Parquet Files in Spark/Scala In this page...

View detail
lite-log scala

Convert String to Date in Spark (Scala)

528 views   0 comments last modified about 5 months ago

Context This pages demonstrates how to convert string to java.util.Date in Spark via Scala. Prerequisites If you have not installed Spark, follow the page below to install it: ...

View detail
zeppelin spark hadoop rdd

Read Text File from Hadoop in Zeppelin through Spark Context

523 views   0 comments last modified about 5 months ago

Background This page provides an example to load text file from HDFS through SparkContext in Zeppelin (sc). Reference The details about this method can be found at: SparkContext.textFile ...

View detail
lite-log spark

Install Spark 2.2.1 in Windows

122 views   0 comments last modified about 5 months ago

This page summarizes the steps to install Spark 2.2.1 in your Windows environment. Tools and Environment GIT Bash Command Prompt Windows 10 Download Binary Package Download the latest binary from the following site: ...

View detail