Hello!

I am a MSc student at the University of Bern. I am working on Debugging Spark Applications for my Master’s Thesis with Haidar Osman.

In 2018, I worked on Strings Attached ^[1][2] for Software Composition Seminar with Haidar Osman.

About my thesis

Debugging is the key to identify errors in a software program. In doing so, critical problems can be avoided. Spark is an engine used to run analyses on large-scale data. Debugging Spark applications is especially significant as no tool, apart from log files, is provided by Spark. However, an application may produce a lengthy log file, which is challenging to examine.

In this research, we have investigated the methods Spark developers use to debug their applications. The problem was introduced by Haidar Osman, who is one of the supervisors for this thesis, as an essential problem in the industry. We have searched for the top frequently asked questions about Spark and reproduced four error logs of three different kinds of exceptions. We held a series of interviews with professional Spark developers. Furthermore, their methodologies were studied to find a pattern of how they track down an error log on a Spark environment.

My first presentation is available on SCG’s website ^[3]

My final presentation is available on SCG’s website ^[4]

[1] http://scg.unibe.ch/wiki/projects/archive/strings-attached

[2] https://github.com/melikegecer/MastersProjects/tree/master/Software_Composition_Seminar

[3] http://scg.unibe.ch/download/softwarecomposition/2019-06-25-Gecer-DebuggingSpark.pdf

[4] http://scg.unibe.ch/download/softwarecomposition/2019-12-10-MelikeGecer-Debugging%20Spark.pdf