<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Data Analysis on Jaehyeon Kim</title><link>https://jaehyeon.me/categories/data-analysis/</link><description>Recent content in Data Analysis on Jaehyeon Kim</description><generator>Hugo -- gohugo.io</generator><language>en</language><copyright>Copyright © 2023-2026 Jaehyeon Kim. All Rights Reserved.</copyright><lastBuildDate>Sat, 30 May 2015 00:00:00 +0000</lastBuildDate><atom:link href="https://jaehyeon.me/categories/data-analysis/index.xml" rel="self" type="application/rss+xml"/><item><title>Setup Random Seeds on Caret Package</title><link>https://jaehyeon.me/blog/2015-05-30-setup-random-seeds-on-caret-package/</link><pubDate>Sat, 30 May 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-05-30-setup-random-seeds-on-caret-package/</guid><description>A short while ago I had a chance to perform analysis using the caret package. One of the requirements is to run it parallelly and to work in both Windows and Linux. The requirement can be met by using the parallel and doParallel packages as the caret package trains a model using the foreach package if clusters are registered by the doParallel package - further details about how to implement parallel processing on a single machine can be found in earlier posts (Link 1, Link 2 and Link 3).</description></item><item><title>Packaging Analysis</title><link>https://jaehyeon.me/blog/2015-03-24-packaging-analysis/</link><pubDate>Tue, 24 Mar 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-03-24-packaging-analysis/</guid><description>When I imagine a workflow, it is performing the same or similar tasks regularly (daily or weekly) in an automated way. Although those tasks can be executed in a script or a *source()*d script, it may not be easy to maintain separate scripts while the size of tasks gets bigger or if they have to be executed in different machines. In academia, reproducible research shares similar ideas but the level of reproducibility introduced in Gandrud, 2013 may not suffice in a business environment as the focus is documenting in a reproducible way.</description></item><item><title>Parallel Processing on Single Machine - Part 3</title><link>https://jaehyeon.me/blog/2015-03-19-parallel-processing-on-single-machine-3/</link><pubDate>Thu, 19 Mar 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-03-19-parallel-processing-on-single-machine-3/</guid><description>Three practical examples are discussed for comparison in this article, concluding this series. In the previous posts, two groups of ways to implement parallel processing on a single machine are introduced. The first group is provided by the snow or parallel package and the functions are an extension of lapply() (LINK). The second group is based on an extension of the for construct (foreach, %dopar% and %:%). The foreach construct is provided by the foreach package while clusters are made and registered by the parallel and doParallel packages respectively (LINK).</description></item><item><title>Parallel Processing on Single Machine - Part 2</title><link>https://jaehyeon.me/blog/2015-03-17-parallel-processing-on-single-machine-2/</link><pubDate>Tue, 17 Mar 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-03-17-parallel-processing-on-single-machine-2/</guid><description>In the previous article, parallel processing on a single machine using the snow and parallel packages are introduced. The four functions are an extension of lapply() with an additional argument that specifies a cluster object. In spite of their effectiveness and ease of use, there may be cases where creating a function that can be sent into clusters is not easy or looping may be more natural. In this article, another way of implementing parallel processing on a single machine is introduced using the foreach and doParallel packages where clusters are created by the parallel package.</description></item><item><title>Parallel Processing on Single Machine - Part 1</title><link>https://jaehyeon.me/blog/2015-03-14-parallel-processing-on-single-machine-1/</link><pubDate>Sat, 14 Mar 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-03-14-parallel-processing-on-single-machine-1/</guid><description>Lack of multi-threading and memory limitation are two outstanding weaknesses of base R. In fact, however, if the size of data is not so large that it can be read in RAM, the former would be relatively easily handled by parallel processing, provided that multiple processors are equipped. This article introduces to a way of implementing parallel processing on a single machine using the snow and parallel packages - the examples are largely based on McCallum and Weston (2012).</description></item><item><title>Tree Based Methods in R - Part 6</title><link>https://jaehyeon.me/blog/2015-03-07-tree-based-methods-6/</link><pubDate>Sat, 07 Mar 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-03-07-tree-based-methods-6/</guid><description>Part I Part II Part III Part IV Part V Part VI (this post) A regression tree is evaluated using bagged trees in the previous article. In this article, the response variable of the same data set is converted into a binary factor variable and a classification tree is evaluated by comparing to bagged trees&amp;rsquo; individual oob/test errors, cumulative oob/test errors and variable importance measures.
Before getting started, note that the source of the classes can be found in this gist and, together with the relevant packages (see tags), it requires a utility function (bestParam()) that can be found here.</description></item><item><title>Tree Based Methods in R - Part 5</title><link>https://jaehyeon.me/blog/2015-03-05-tree-based-methods-5/</link><pubDate>Thu, 05 Mar 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-03-05-tree-based-methods-5/</guid><description>Part I Part II Part III Part IV Part V (this post) Part VI This article evaluates a single regression tree&amp;rsquo;s performance by comparing to bagged trees&amp;rsquo; individual oob/test errors, cumulative oob/test errors and variable importance measures.
Before getting started, note that the source of the classes can be found in this gist and, together with the relevant packages (see tags), it requires a utility function (bestParam()) that can be found here.</description></item><item><title>Tree Based Methods in R - Part 4</title><link>https://jaehyeon.me/blog/2015-02-15-tree-based-methods-4/</link><pubDate>Sun, 15 Feb 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-02-15-tree-based-methods-4/</guid><description>Part I Part II Part III Part IV (this post) Part V Part VI While the last three articles illustrated the CART model for both classification (with equal/unequal costs) and regression tasks, this article is rather technical as it compares three packages: rpart, caret and mlr. For those who are not familiar with the last two packages, they are wrappers (or frameworks) that implement a range of models (or algorithms) in a unified way.</description></item><item><title>Tree Based Methods in R - Part 3</title><link>https://jaehyeon.me/blog/2015-02-14-tree-based-methods-3/</link><pubDate>Sat, 14 Feb 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-02-14-tree-based-methods-3/</guid><description>Part I Part II Part III (this post) Part IV Part V Part VI While classification tasks are implemented in the last two articles (Part I and Part II), a regression task is the topic of this article. While the caret package selects the tuning parameter (cp) that minimizes the error (RMSE), the rpart packages recommends the 1-SE rule, which selects the smallest tree within 1 standard error of the minimum cross validation error (xerror).</description></item><item><title>Tree Based Methods in R - Part 2</title><link>https://jaehyeon.me/blog/2015-02-08-tree-based-methods-2/</link><pubDate>Sun, 08 Feb 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-02-08-tree-based-methods-2/</guid><description>Part I Part II (this post) Part III Part IV Part V Part VI In the previous article (Tree Based Methods in R - Part I), a decision tree is created on the Carseats data which is in the chapter 8 lab of ISLR. In that article, potentially asymetric costs due to misclassification are not taken into account. When unbalance between false positive and false negative can have a significant impact, it can be explicitly adjusted either by altering prior (or empirical) probabilities or by adding a loss matrix.</description></item><item><title>Tree Based Methods in R - Part 1</title><link>https://jaehyeon.me/blog/2015-02-01-tree-based-methods-1/</link><pubDate>Sun, 01 Feb 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-02-01-tree-based-methods-1/</guid><description>Part I (this post) Part II Part III Part IV Part V Part VI This is the first article about tree based methods using R. Carseats data in the chapter 8 lab of ISLR is used to perform classification analysis. Unlike the lab example, the rpart package is used to fit the CART model on the data and the caret package is used for tuning the pruning parameter (cp).</description></item><item><title>Quick Trial of Adding Column</title><link>https://jaehyeon.me/blog/2015-01-14-quick-trial-of-adding-column/</link><pubDate>Wed, 14 Jan 2015 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2015-01-14-quick-trial-of-adding-column/</guid><description>This is a quick trial of adding overall and conditional (by user) average columns in a data frame. base,plyr,dplyr,data.table,dplyr + data.table packages are used. Personally I perfer dplyr + data.table - dplyr for comperhensive syntax and data.table for speed.
1## set up variables 2size &amp;lt;- 36000 3numUsers &amp;lt;- 4900 4# roughly each user has 7 sessions 5numSessions &amp;lt;- (numUsers / 7) - ((numUsers / 7) %% 1) 6 7## create data frame 8set.</description></item><item><title>Looping without for</title><link>https://jaehyeon.me/blog/2014-12-17-looping-without-for/</link><pubDate>Wed, 17 Dec 2014 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2014-12-17-looping-without-for/</guid><description>Purely programming point of view, I consider for-loops would be better to be avoided in R as
the script can be more readable it is easier to handle errors Some articles on the web indicate that looping functions (or apply family of functions) don&amp;rsquo;t guarantee faster execution and sometimes even slower. Although, assuming that the experiments are correct, in my opinion, code readability itself is beneficial enough to avoid for-loops. Even worse, R&amp;rsquo;s dynamic typing system coupled with poor readability can result in a frustrating consequence as the code grows.</description></item><item><title>Short R Examples</title><link>https://jaehyeon.me/blog/2014-12-03-short-r-examples/</link><pubDate>Wed, 03 Dec 2014 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2014-12-03-short-r-examples/</guid><description>As I don&amp;rsquo;t use R at work yet, I haven&amp;rsquo;t got enough chances to learn R by doing. Together with teaching myself machine learning with R, I consider it would be a good idea to collect examples on the web. Recently I have been visiting a Linkedin group called The R Project for Statistical Computing and suggesting scripts on data manipulation or programming topics. Actually I expected some of the topics may also be helpful to learn machine learning/statistics but, possibly due to lack of understaning of the topics in context, I&amp;rsquo;ve found only a few - it may be necessary to look for another source.</description></item><item><title>Summarise Stock Returns from Multiple Files</title><link>https://jaehyeon.me/blog/2014-11-27-summarise-stock-returns-from-multiple-files/</link><pubDate>Thu, 27 Nov 2014 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2014-11-27-summarise-stock-returns-from-multiple-files/</guid><description>Gross returns, standard deviation and correlation of multiple shares are produced here. This post is a slight extension of the previous two articles (Download Stock Data - Part I, Download Stock Data - Part II).
Status, September 2026. Google retired the finance/historical CSV endpoint that the download step uses, so the price files this post summarises can no longer be fetched. The return, standard deviation and correlation steps still hold for any price files that carry Date, Close and Code columns.</description></item><item><title>Download Stock Data - Part II</title><link>https://jaehyeon.me/blog/2014-11-21-download-stock-data-2/</link><pubDate>Fri, 21 Nov 2014 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2014-11-21-download-stock-data-2/</guid><description>In an earlier article, a way to download stock price data files from Google, save it into a local drive and merge them into a single data frame. If files are not large, however, it wouldn&amp;rsquo;t be effective and, in this article, files are downloaded and merged internally.
Status, September 2026. Google retired the finance/historical CSV endpoint that the URLs here point at, so reading those URLs no longer returns data.</description></item><item><title>Download Stock Data - Part I</title><link>https://jaehyeon.me/blog/2014-11-20-download-stock-data-1/</link><pubDate>Thu, 20 Nov 2014 00:00:00 +0000</pubDate><guid>https://jaehyeon.me/blog/2014-11-20-download-stock-data-1/</guid><description>Stock price data files are downloaded from Google, saved into a local drive and merged into a single data frame. This script is slightly modified from a script which downloads RStudio package download log data. The original source can be found here.
Status, September 2026. Google retired the finance/historical CSV endpoint that this script downloads from, so the download step no longer returns data. The folder creation, error handling and merge patterns still hold if you point them at a price source that is still published.</description></item></channel></rss>