Threats to Validity

Author: swissmanu

paper.tex

@@ -115,7 +115,7 @@ \subsection{ReactiveX and RxJS}
 \section{Interviews and War Stories}
 \label{sec:interviews}
 
-We have worked together with 15 professional software engineers during the process of answering RQ1. On the way of finding our interview partners and war stories reporters, we noticed it to be a challenge on its own to find people which understand themselves as users of RP and related technologies. E.g. even though Angular makes heavy use of RxJS, we will see that many engineers do not interact with its abstractions directly in order to build simple UIs. In the end we were able to gather interesting statements nonetheless which we are going to present in this section.
+We have worked together professional software engineers during the process of answering RQ1. On the way of finding our interview partners and war stories reporters, we noticed it to be a challenge on its own to find people which understand themselves as users of RP and related technologies. E.g. even though Angular makes heavy use of RxJS, we will see that many engineers do not interact with its abstractions directly in order to build simple UIs. In the end we were able to gather interesting statements nonetheless which we are going to present in this section.
 
 \subsection{Informal Interviews}
 
@@ -325,7 +325,7 @@ \subsection{Integration}
 
 75\% of our observational study's participants used the debugging tools integrated to their browser and/or IDE. Even though some of them stated to know additional external utilities in the after-action survey, none of them went that far and used them during the one-hour session.
 
-We understand the observed behavior as evidence for a hurdle preventing engineers from using specialized RP debugging utilities. In order to reach more software engineers with RP domain-specific debugging tools, we want to provide a potential solution as tightly integrated with the developer tools of the browser and or an IDE.
+We understand the observed behavior as evidence for a hurdle preventing engineers from using specialized RP debugging utilities. In order to reach more software engineers with RP domain-specific debugging tools, we want to provide a potential solution as tightly integrated with the developer tools of the browser and/or an IDE as possible.
 
 \subsection{Other Areas of Improvement}
 
@@ -334,33 +334,18 @@ \subsection{Other Areas of Improvement}
 \section{Threats to Validity}
 \label{sec:threats}
 
-\begin{itemize}
-	\item Informal interviews/war stories only
-	\begin{itemize}
-		\item Describe background of interview partners
-		\item Only 4 Participants in experiment
-		\item Code in experiment carefully crafted... Though is it representative?
-		\
-	\end{itemize}
-\end{itemize}
+We worked with 15 professional software engineers while writing this paper. Six interview partners and five war story reports helped us building the foundation for a later observational study with a total of four participants.
 
+While we claim nine interviewees and reporters to have been an appropriate number of peers for collecting the initial data basis, we are aware that our four subjects for the observational study lead to a result lacking significance.
 
-\begin{itemize}
-	\item Pick up results from war stories and experiment
-	\item Propose one solution per challenges
-	\item Sketch how this could look like
-	\item This answers RQ2
-\end{itemize}
+More time given, we suggest to conduct the study with a bigger population again to obtain a result with higher significance and providing more insight in how engineers debug RxJS-based code. Doing so would possibly require a statistical evaluation of these results.
+
+Furthermore, this was the first time we designed an observational study of this kind. Though we carefully peer-reviewed the study design with our advisor, we cannot claim the design and its execution to be flawless. E.g. we would use a commonly known business domain for both problems presented to the subjects. This would allow them to understand the functionality of the application in front of them quicker and more focus on resolving the task given.
 
 \section{Conclusion}
 \label{sec:conclusion}
 
-\begin{itemize}
-	\item Recap Past Developments in RP Debugging
-	\item Recap proposed solution
-	\item Whats next in PA2?
-	\item Whats the vision for the thesis?
-\end{itemize}
+
 
 
 \bibliographystyle{splncs04}