<?xml version="1.0" encoding="UTF-8"?>
<rss  xmlns:atom="http://www.w3.org/2005/Atom" 
      xmlns:media="http://search.yahoo.com/mrss/" 
      xmlns:content="http://purl.org/rss/1.0/modules/content/" 
      xmlns:dc="http://purl.org/dc/elements/1.1/" 
      version="2.0">
<channel>
<title>José Caro</title>
<link>https://carobarrera.com/atom.html</link>
<atom:link href="https://carobarrera.com/atom.xml" rel="self" type="application/rss+xml"/>
<description>José Caro is an assistant professor at Córdoba University, researcher and teaching statistics.</description>
<generator>quarto-1.8.25</generator>
<lastBuildDate>Mon, 28 Sep 2026 22:00:00 GMT</lastBuildDate>
<item>
  <title>Mapping the Miles: A Running Log as a Calendar Heatmap</title>
  <dc:creator>José Caro</dc:creator>
  <link>https://carobarrera.com/blog/2026/09/</link>
  <description><![CDATA[ 





<section id="the-loneliness-of-the-long-distance-runner" class="level2">
<h2 class="anchored" data-anchor-id="the-loneliness-of-the-long-distance-runner">The loneliness of the long distance runner</h2>
<p>I’m not much of a Strava person, and I’ve never been keen on the apps that upload your training to a social network to share with other runners. Still, I like to keep track of my daily workouts, and I do it the old-fashioned way: a plain Excel file (<code>running.xlsx</code>) with one row per day and a column for the kilometers logged.</p>
<p>If you’re also into data visualization and work with it professionally, though, a plain spreadsheet is never quite enough — you end up wanting to <em>see</em> the training rather than just log it. A calendar heatmap turns out to be a simple but visually powerful way to look at training volume week by week, month by month, across the whole year: darker tiles mean longer runs, lighter tiles mean rest or short recovery days, and the empty gaps are immediately obvious in a way that a spreadsheet row never quite shows.</p>
<section id="building-the-heatmap" class="level3">
<h3 class="anchored" data-anchor-id="building-the-heatmap">Building the heatmap</h3>
<p>The script below (<code>training.R</code>) reads the <code>2025</code> sheet from <code>running.xlsx</code>, cleans up the dates and distances, and then lays each day out on a small monthly calendar grid faceted by month. Let’s walk through it piece by piece.</p>
<p><strong>1. Load the libraries and read the log.</strong> Nothing fancy here — <code>readxl</code> pulls the spreadsheet in, <code>dplyr</code> and <code>lubridate</code> do the date wrangling, and <code>ggplot2</code> draws the thing.</p>
<div class="code-with-filename">
<div class="code-with-filename-file">
<pre><strong>training.R</strong></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb1" data-filename="training.R" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb1-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(ggplot2)</span>
<span id="cb1-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(dplyr)</span>
<span id="cb1-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(lubridate)</span>
<span id="cb1-4"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(scales)</span>
<span id="cb1-5"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(readxl)  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># For reading Excel files</span></span>
<span id="cb1-6"></span>
<span id="cb1-7"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Read the Excel file</span></span>
<span id="cb1-8">kms <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">read_excel</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"blog/2026/09/running.xlsx"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sheet =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"2025"</span>)</span></code></pre></div></div>
</div>
<p><strong>2. Clean the data and print a quick summary.</strong> Dates get coerced properly, rows with missing values are dropped, and distances are forced to numeric. Nothing here should ever trigger, but a training log accumulated by hand over a year <em>will</em> eventually have a typo’d cell.</p>
<div class="code-with-filename">
<div class="code-with-filename-file">
<pre><strong>training.R</strong></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb2" data-filename="training.R" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb2-1">kms <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> kms <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb2-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">date =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">as.Date</span>(date)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb2-3">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">!</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">is.na</span>(date), <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">!</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">is.na</span>(kms)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb2-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">kms =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">as.numeric</span>(kms)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb2-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">filter</span>(kms <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">&gt;=</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">0</span>)</span>
<span id="cb2-6"></span>
<span id="cb2-7"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cat</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Data Summary:</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span>)</span>
<span id="cb2-8"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cat</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Date range:"</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">as.character</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">min</span>(kms<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>date)), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"to"</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">as.character</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">max</span>(kms<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>date)), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span>)</span>
<span id="cb2-9"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cat</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Total days:"</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">nrow</span>(kms), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span>)</span>
<span id="cb2-10"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cat</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Total kilometers:"</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span>(kms<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>kms, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"km</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span>)</span>
<span id="cb2-11"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">cat</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Average daily kilometers:"</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">round</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(kms<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>kms, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>), <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"km</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span>)</span></code></pre></div></div>
</div>
<p><strong>3. Work out where each day sits on its monthly calendar grid.</strong> <code>ggplot2</code> has no built-in calendar geometry, so this is the part that actually earns its keep: for every date we compute its weekday (Monday-first) and its <code>week_of_month</code>, correcting for the fact that the 1st of the month rarely falls on a Monday. Each day also gets tagged with that month’s running total, which ends up in the facet label.</p>
<div class="code-with-filename">
<div class="code-with-filename-file">
<pre><strong>training.R</strong></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb3" data-filename="training.R" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb3-1">create_kms_calendar_heatmap <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(data, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Daily Kilometers Calendar Heatmap"</span>) {</span>
<span id="cb3-2"></span>
<span id="cb3-3">  calendar_data_prep <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> data <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-4">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">rename</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">value =</span> kms)</span>
<span id="cb3-5"></span>
<span id="cb3-6">  monthly_totals <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> calendar_data_prep <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">year =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">year</span>(date), <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">month =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">month</span>(date)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-8">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">group_by</span>(year, month) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-9">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">summarise</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">total_kms =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span>(value, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>), <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">.groups =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'drop'</span>)</span>
<span id="cb3-10"></span>
<span id="cb3-11">  calendar_data <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> calendar_data_prep <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-12">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(</span>
<span id="cb3-13">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">year =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">year</span>(date),</span>
<span id="cb3-14">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">month =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">month</span>(date),</span>
<span id="cb3-15">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">day =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">day</span>(date),</span>
<span id="cb3-16">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">weekday =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">wday</span>(date, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">week_start =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>), <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Monday = 1</span></span>
<span id="cb3-17">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">month_name =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">month</span>(date, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">label =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">abbr =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>)</span>
<span id="cb3-18">    ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-19">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">group_by</span>(year, month) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-20">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(</span>
<span id="cb3-21">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">first_day_of_month =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">floor_date</span>(date, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"month"</span>),</span>
<span id="cb3-22">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">first_weekday =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">wday</span>(first_day_of_month, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">week_start =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>),</span>
<span id="cb3-23">      <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Offset by how far into the week the 1st of the month falls</span></span>
<span id="cb3-24">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">week_of_month =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ceiling</span>((day <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span> first_weekday <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">7</span>)</span>
<span id="cb3-25">    ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-26">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ungroup</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-27">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">left_join</span>(monthly_totals, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">by =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"year"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"month"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-28">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(</span>
<span id="cb3-29">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">month_year_label =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">paste0</span>(month_name, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">" "</span>, year, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">\n</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">("</span>, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">round</span>(total_kms, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">" km)"</span>),</span>
<span id="cb3-30">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">month_year_date =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">as.Date</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">paste</span>(year, month, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"01"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"-"</span>))</span>
<span id="cb3-31">    ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-32">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">arrange</span>(month_year_date)</span>
<span id="cb3-33"></span>
<span id="cb3-34">  month_order <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> calendar_data <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-35">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">select</span>(month_year_label, month_year_date) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-36">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">distinct</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-37">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">arrange</span>(month_year_date) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb3-38">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">pull</span>(month_year_label)</span>
<span id="cb3-39"></span>
<span id="cb3-40">  calendar_data<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>month_year_label <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">factor</span>(calendar_data<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>month_year_label, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">levels =</span> month_order)</span>
<span id="cb3-41"></span>
<span id="cb3-42">  calendar_data</span>
<span id="cb3-43">}</span></code></pre></div></div>
</div>
<p><strong>4. Draw the tiles.</strong> With the calendar grid in place, the plot itself is a straightforward <code>geom_tile()</code> faceted by month, colored on a diverging scale centered on the median distance — so an easy 5K and a 25K long run stand out on opposite ends, and typical training days fade into the gray middle.</p>
<div class="code-with-filename">
<div class="code-with-filename-file">
<pre><strong>training.R</strong></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb4" data-filename="training.R" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb4-1">plot_kms_calendar <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="cf" style="color: #003B4F;
background-color: null;
font-weight: bold;
font-style: inherit;">function</span>(calendar_data, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Daily Kilometers Calendar Heatmap"</span>) {</span>
<span id="cb4-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(calendar_data, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> weekday, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span>week_of_month, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">fill =</span> value)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-3">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_tile</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">color =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"white"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">linewidth =</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.5</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-4">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">facet_wrap</span>(<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> month_year_label, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">ncol =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">scales =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"free_y"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-5">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_x_continuous</span>(</span>
<span id="cb4-6">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">breaks =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span><span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">:</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">7</span>,</span>
<span id="cb4-7">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labels =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">c</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Mon"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Tue"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Wed"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Thu"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Fri"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Sat"</span>, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Sun"</span>),</span>
<span id="cb4-8">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">position =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"top"</span></span>
<span id="cb4-9">    ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-10">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_y_continuous</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">breaks =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">NULL</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-11">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_fill_gradient2</span>(</span>
<span id="cb4-12">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">low =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"lightgray"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">mid =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"yellow"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">high =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"darkred"</span>,</span>
<span id="cb4-13">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">midpoint =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">median</span>(calendar_data<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>value, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>),</span>
<span id="cb4-14">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">name =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Kilometers"</span></span>
<span id="cb4-15">    ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-16">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">labs</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">title =</span> title, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">""</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">""</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-17">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme_minimal</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb4-18">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">theme</span>(</span>
<span id="cb4-19">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">axis.text.y =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">element_blank</span>(),</span>
<span id="cb4-20">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">axis.text.x.bottom =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">element_blank</span>(),</span>
<span id="cb4-21">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">axis.ticks =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">element_blank</span>(),</span>
<span id="cb4-22">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">panel.grid =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">element_blank</span>(),</span>
<span id="cb4-23">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">strip.background =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">element_blank</span>(),</span>
<span id="cb4-24">      <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">legend.position =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"bottom"</span></span>
<span id="cb4-25">    )</span>
<span id="cb4-26">}</span>
<span id="cb4-27"></span>
<span id="cb4-28">kms_calendar_plot <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">plot_kms_calendar</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">create_kms_calendar_heatmap</span>(kms))</span>
<span id="cb4-29"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">print</span>(kms_calendar_plot)</span></code></pre></div></div>
</div>
<p><strong>5. A colorblind-friendly variant, and the monthly numbers behind the chart.</strong> Swapping in the <code>viridis</code> “plasma” scale is a one-liner. And since the heatmap is meant to replace staring at a spreadsheet, it’s only fair to also print the spreadsheet-style summary it’s replacing.</p>
<div class="code-with-filename">
<div class="code-with-filename-file">
<pre><strong>training.R</strong></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode" id="cb5" data-filename="training.R" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb5-1">kms_calendar_viridis <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">plot_kms_calendar</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">create_kms_calendar_heatmap</span>(kms), <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Daily Kilometers"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb5-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_fill_viridis_c</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">name =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Kilometers"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">option =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"plasma"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">trans =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"sqrt"</span>)</span>
<span id="cb5-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">print</span>(kms_calendar_viridis)</span>
<span id="cb5-4"></span>
<span id="cb5-5">monthly_stats <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> kms <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb5-6">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">year_month =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">floor_date</span>(date, <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"month"</span>)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb5-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">group_by</span>(year_month) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb5-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">summarise</span>(</span>
<span id="cb5-9">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">total_kms =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">sum</span>(kms, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>),</span>
<span id="cb5-10">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">avg_daily_kms =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mean</span>(kms, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>),</span>
<span id="cb5-11">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">max_daily_kms =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">max</span>(kms, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">na.rm =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>),</span>
<span id="cb5-12">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">days_with_data =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">n</span>(),</span>
<span id="cb5-13">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">.groups =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">'drop'</span></span>
<span id="cb5-14">  )</span>
<span id="cb5-15"></span>
<span id="cb5-16"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">print</span>(monthly_stats)</span></code></pre></div></div>
</div>
<p>And here’s the result for 2025:</p>
<p><img src="https://carobarrera.com/blog/2026/09/heatmap.png" class="img-fluid"></p>
<p>The gaps are the honest part of the chart — the weeks I meant to run and didn’t are just as visible as the weeks I did. All told, 2025 added up to <strong>5,310 km</strong> — enough to have jogged from Córdoba to Ulaanbaatar, if I’d only picked a less roundabout route than “out my front door and back.”</p>


<!-- -->

</section>
</section>

 ]]></description>
  <category>heatmap</category>
  <category>data viz</category>
  <category>running</category>
  <guid>https://carobarrera.com/blog/2026/09/</guid>
  <pubDate>Mon, 28 Sep 2026 22:00:00 GMT</pubDate>
  <media:content url="https://carobarrera.com/blog/2026/09/heatmap.png" medium="image" type="image/png" height="101" width="144"/>
</item>
<item>
  <title>Understanding Linear Regression</title>
  <dc:creator>José Caro</dc:creator>
  <link>https://carobarrera.com/blog/2023/05/regression/</link>
  <description><![CDATA[ 





<div class="goals">
<div class="goals-header">
<p>Learning goals</p>
</div>
<div class="goals-container">
<ul>
<li>To learn how to apply linear regression models in practice.</li>
<li>To identify the predictor and the reponse variables.</li>
<li>To interpret estimates and diagnostic statistics.</li>
</ul>
</div>
</div>
<div class="callout callout-style-default callout-important callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Important</span>Better to remember some of the concepts already studied!
</div>
</div>
<div class="callout-body-container callout-body">
<p>Have a look at the network graph, see the the complex intrincacies of the regression analysis and check out if you remember some of the concepts (in the red boxes) we have already studied. Don’t panic! You do not have to know all of them and you won’t study the most of them, just keep in mind some of the concepts (which means review them) for the lecture.</p>
<p><img src="https://carobarrera.com/blog/2023/05/regression/network.png" class="img-fluid"></p>
</div>
</div>
<section id="what-is-regression-analysis" class="level2">
<h2 class="anchored" data-anchor-id="what-is-regression-analysis">What is Regression Analysis?</h2>
<!-- [Referencia 1:](https://deasadiqbal.medium.com/demystifying-machine-learning-a-guided-tour-of-the-top-10-algorithms-3fee19d6c2a8)
[Referencia 2:](https://medium.com/analysts-corner/mastering-regression-analysis-from-basics-to-advanced-applications-in-python-and-r-532e100e0fa0)
[Referencia 3:](https://medium.com/the-stata-gallery/correlation-vs-regression-a-key-difference-that-many-analysts-miss-3770c9b368d9)
-->
<p>Regression analysis is a statistical method used <strong>to predict the value of a dependent variable based on the values of one or more independent variables</strong>. It involves analyzing the relationship between the dependent variable and the independent variables to understand how changes in the independent variables affect the dependent variable.</p>
<p>The independent variables, also known as <em>predictors</em>, are used to estimate or “predict” the value of the <em>dependent</em> variable. The relationship between the variables is typically represented by an equation or a mathematical model. The regression model estimates the relationship by fitting a line (or curve) through the data points, allowing for the prediction of the dependent variable for given values of the independent variables.</p>
<p>Regression analysis can be used to measure the influence of one or multiple variables on another variable. By identifying the strength and direction of the relationship, regression analysis helps researchers and analysts to understand the underlying factors that impact the dependent variable. It is a powerful tool for making predictions, forecasting future outcomes, and understanding the complex mechanisms driving the variables of interest.</p>
<p>Regression analysis finds applications in various fields such as economics, finance, social sciences, and healthcare. In economics, it can be used to analyze the relationship between factors like income, inflation, and consumer spending. In finance, it aids in predicting stock prices and identifying factors that impact investment returns. In the social sciences, regression analysis is used to study the impact of variables like education, income, and demographic characteristics on various outcomes. In healthcare, it helps in understanding the factors influencing patient outcomes and optimizing treatment strategies.</p>
<!--
Regression analysis provides valuable insights into the relationship between variables and helps to make informed decisions. By examining the statistical significance of the independent variables, researchers can determine whether a particular variable has a significant impact on the dependent variable. The coefficients of the independent variables provide information about the direction and magnitude of the relationship.-->
<p>Regression analysis can also assist in model validation and hypothesis testing. Researchers can develop different regression models to test alternative hypotheses and evaluate the significance of each variable. The quality of the regression model can be assessed by examining various statistical measures such as <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"> and <em>p</em>-values. These measures provide information about the <em>goodness-of-fit</em> of the model and the <em>statistical significance</em> of the coefficients.</p>
<!--
Overall, regression analysis is a versatile and widely used statistical technique that helps in understanding the relationship between variables and making predictions or forecasts. By providing a quantitative approach to analyzing data, regression analysis enhances decision-making and enables researchers to draw meaningful conclusions from complex datasets. Its applications span across various disciplines and play a vital role in shaping our understanding of the world around us.-->
<p>In the field of statistics, regression analysis is a widely used method for modeling the relationship between a dependent variable and one or more independent variables. It helps in understanding the nature and strength of the relationship between variables, making predictions, and drawing inferences. There are various types of regression analysis, each suitable for different scenarios and data types. In this post, we will explore the simple and multiple linear regression, however, another common types of regression models are: logistic regression, ordinal regression, and nominal regression among others.</p>
</section>
<section id="the-motive-behind-linear-regression" class="level2">
<h2 class="anchored" data-anchor-id="the-motive-behind-linear-regression">The motive behind linear regression</h2>
<p>Linear regression is useful when we suspect a <strong>linear relationship</strong> between variables (known as explanatory variables, predictors, or covariates) and a response variable. While it may seem straightforward, linear regression is a powerful and widely used statistical learning method.</p>
<section id="simple-linear-regression" class="level3">
<h3 class="anchored" data-anchor-id="simple-linear-regression">Simple Linear Regression</h3>
<div class="note">
<p><strong>Simple linear regression</strong> is the most basic form of regression analysis, where a single independent variable is used to predict the value of the dependent variable. The relationship between the independent variable <img src="https://latex.codecogs.com/png.latex?x"> and dependent variable <img src="https://latex.codecogs.com/png.latex?y"> is assumed to be linear, following a straight line equation: <img src="https://latex.codecogs.com/png.latex?y=%5Calpha+%5Cbeta%5Ccdot%20x+%5Cvarepsilon"> Note that the notation may also be given as: <img src="https://latex.codecogs.com/png.latex?%20y=%5Cbeta_%7B0%7D%20+%20%5Cbeta_%7B1%7D%5Ccdot%20x+%5Cvarepsilon,"> being both equivalent.</p>
<p>Here, the coefficients <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B0%7D"> and <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B1%7D"> are the intercept and slope coefficients, respectively, and <img src="https://latex.codecogs.com/png.latex?%5Cvarepsilon"> represents the random error term. The goal is to estimate the coefficients that provide the best fit line to the data.</p>
</div>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Tip</span>Worth to remember!
</div>
</div>
<div class="callout-body-container callout-body">
<p>This type of regression is useful when exploring the relationship between two continuous variables. For example, we can use simple linear regression to predict the sales of a product based on its price or to understand the impact of study hours on exam scores. By analyzing the slope and intercept coefficients, we can determine the direction and strength of the relationship.</p>
</div>
</div>
<!--
::: {.panel-tabset .nav-pills}
## Scatterplot

Content for first tab

## Boxplot

Content for second tab

## Barplot

Content for third tab
:::
--->
<section id="example" class="level4">
<h4 class="anchored" data-anchor-id="example">Example</h4>
<p>Lets see the following example with the <strong><em>Advertising</em></strong> dataset. Data can be downloaded in <code>.csv</code> format from <a href="https://www.kaggle.com/datasets/bumba5341/advertisingcsv/">kaggle</a>. It represents the data sales (in thousands of units) for a particular product advertising budgets (in thousands of dollars) for TV, radio, and newspaper media:</p>
<div class="cell">
<div class="cell-output-display">
<table class="caption-top table table-sm table-striped small">
<thead>
<tr class="header">
<th style="text-align: right;">X</th>
<th style="text-align: right;">TV</th>
<th style="text-align: right;">Radio</th>
<th style="text-align: right;">Newspaper</th>
<th style="text-align: right;">Sales</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: right;">1</td>
<td style="text-align: right;">230.1</td>
<td style="text-align: right;">37.8</td>
<td style="text-align: right;">69.2</td>
<td style="text-align: right;">22.1</td>
</tr>
<tr class="even">
<td style="text-align: right;">2</td>
<td style="text-align: right;">44.5</td>
<td style="text-align: right;">39.3</td>
<td style="text-align: right;">45.1</td>
<td style="text-align: right;">10.4</td>
</tr>
<tr class="odd">
<td style="text-align: right;">3</td>
<td style="text-align: right;">17.2</td>
<td style="text-align: right;">45.9</td>
<td style="text-align: right;">69.3</td>
<td style="text-align: right;">9.3</td>
</tr>
<tr class="even">
<td style="text-align: right;">4</td>
<td style="text-align: right;">151.5</td>
<td style="text-align: right;">41.3</td>
<td style="text-align: right;">58.5</td>
<td style="text-align: right;">18.5</td>
</tr>
<tr class="odd">
<td style="text-align: right;">5</td>
<td style="text-align: right;">180.8</td>
<td style="text-align: right;">10.8</td>
<td style="text-align: right;">58.4</td>
<td style="text-align: right;">12.9</td>
</tr>
<tr class="even">
<td style="text-align: right;">6</td>
<td style="text-align: right;">8.7</td>
<td style="text-align: right;">48.9</td>
<td style="text-align: right;">75.0</td>
<td style="text-align: right;">7.2</td>
</tr>
</tbody>
</table>
</div>
</div>
<p>We want to estimate the product sales based on TV advertising budget. Based on this we identify the following: <img src="https://latex.codecogs.com/png.latex?y=%5Cbeta_%7B0%7D%20+%20%5Cbeta_%7B1%7D%5Ccdot%20x+%5Cvarepsilon"> Which in our case, the dependent (or response) variable, <img src="https://latex.codecogs.com/png.latex?y">, is the <em>sales</em> and the independent (or explanatory) variable, <img src="https://latex.codecogs.com/png.latex?x">, is the <em>TV advertising budget</em>, thus we can establish: <img src="https://latex.codecogs.com/png.latex?Sales=%5Cbeta_%7B0%7D+%5Cbeta_%7B1%7D%5Ccdot%20TV+%5Cvarepsilon"></p>
<p>The scatterplot shows the following visualization of our model with the values for the <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B0%7D"> and <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B1%7D"> coefficients, which are: <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B0%7D=7.03"> and <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B1%7D=0.05"></p>
<div class="cell">
<div class="cell-output cell-output-stdout">
<pre><code>Matplotlib is building the font cache; this may take a moment.</code></pre>
</div>
<div class="cell-output cell-output-stdout">
<pre><code>(-10.0, 310.0)</code></pre>
</div>
<div class="cell-output cell-output-stdout">
<pre><code>(0.0, 28.27)</code></pre>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2023/05/regression/index_files/figure-html/unnamed-chunk-2-1.png" class="img-fluid figure-img" width="672"></p>
</figure>
</div>
</div>
</div>
<section id="interpretation-of-the-coefficients-of-the-model" class="level5">
<h5 class="anchored" data-anchor-id="interpretation-of-the-coefficients-of-the-model">Interpretation of the coefficients of the model</h5>
<p>Once the model has been specified: <img src="https://latex.codecogs.com/png.latex?Sales=%5Coverbrace%7B7.03%7D%5E%7B%5Cbeta_%7B0%7D%7D+%5Coverbrace%7B0.05%7D%5E%7B%5Cbeta_%7B1%7D%7D%5Ccdot%20TV+%5Cvarepsilon"></p>
<p>The coefficient, <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B1%7D=0.05">, represents the average difference in the <em>Sales</em> for one-unit difference in the <img src="https://latex.codecogs.com/png.latex?TV%5C%20budget">. In other words, we expect each additional euro (unitary money) spent in TV budget, on average, to raise the sales by <img src="https://latex.codecogs.com/png.latex?0.05">.</p>
<p>The intercept, <img src="https://latex.codecogs.com/png.latex?%5Cbeta_%7B0%7D=7.03">, represents the predicted <em>Sales</em> when <img src="https://latex.codecogs.com/png.latex?TV%5C%20budget=0">, that is, it represents the average sales of a zero TV budget. Because this value doesn’t make much intuitive sense, it’s common for models to be transformed and standardized before carrying out a regression model.</p>
</section>
</section>
</section>
<section id="multiple-linear-regression" class="level3">
<h3 class="anchored" data-anchor-id="multiple-linear-regression">Multiple Linear Regression</h3>
<div class="note">
<p>Unlike simple linear regression, multiple linear regression involves using multiple independent variables to predict the value of the dependent variable. The relationship can be expressed as: <img src="https://latex.codecogs.com/png.latex?y=%5Cbeta_%7B0%7D+%5Cbeta_%7B1%7Dx_%7B1%7D+%5Cbeta_%7B2%7Dx_%7B2%7D+%5Cldots+%5Cbeta_%7Bi%7Dx_%7Bi%7D+%5Cvarepsilon"></p>
<p>where <img src="https://latex.codecogs.com/png.latex?i"> is the number of predictors. The goal is to estimate the coefficients that provide the best-fitting hyperplane in the <em>p</em>-dimensional space.</p>
</div>
<div class="callout callout-style-default callout-tip callout-titled">
<div class="callout-header d-flex align-content-center">
<div class="callout-icon-container">
<i class="callout-icon"></i>
</div>
<div class="callout-title-container flex-fill">
<span class="screen-reader-only">Tip</span>Worth to remember!
</div>
</div>
<div class="callout-body-container callout-body">
<p>This type of regression analysis is suitable when dealing with more complex relationships and multiple factors influencing the dependent variable. For example, in market research, we might use multiple linear regression to predict consumer spending based on factors like income, age, and education level. By considering multiple variables simultaneously, we can gain deeper insights into the factors driving the outcome.</p>
</div>
</div>
<section id="example-1" class="level4">
<h4 class="anchored" data-anchor-id="example-1">Example</h4>
<p>In previous example lets assume now that we want to estimate the product sales based on TV and Radio advertising budget. Hence we specify the following model: <img src="https://latex.codecogs.com/png.latex?y%20=%20%5Cbeta_%7B0%7D+%5Cbeta_%7B1%7D%5Ccdot%20x_%7B1%7D+%5Cbeta_%7B2%7D%5Ccdot%20x_%7B2%7D+%5Cvarepsilon"> Note that we do not use the notation <img src="https://latex.codecogs.com/png.latex?%5Calpha"> for the intercept as in the simple model this is because in that model the notation is similar to the one</p>
<div class="cell">
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2023/05/regression/index_files/figure-html/unnamed-chunk-3-3.png" class="img-fluid figure-img" width="1440"></p>
</figure>
</div>
</div>
</div>
</section>
</section>
<section id="assumptions-of-the-linear-model" class="level3">
<h3 class="anchored" data-anchor-id="assumptions-of-the-linear-model">Assumptions of the linear model</h3>
<p>The linear regression performs well is the following assumptions are made:</p>
<p><strong>1. Linearity:</strong> There is a linear relationship between the predictors and the response variable. That is, the deterministic component of a regression model is a linear function of the separate predictors. You can use scatterplots to visually verify this.</p>
<p><strong>2. Independence of errors:</strong> It means that the value of one error does not predict the value of another error. This is crucial for the reliability of standard errors, confidence intervals, and hypothesis tests.</p>
<p><strong>3. Constant variance (homoscedasticity):</strong> This means that the residuals have the same variance for every value of the fitted values and of the predictors. One way to detect it would be using tests and plotting residuals.</p>
</section>
</section>
<section id="testing-the-significance-of-a-regression" class="level2">
<h2 class="anchored" data-anchor-id="testing-the-significance-of-a-regression">Testing the significance of a regression</h2>
<p>There are several ways the significance of a regression can be tested. Providing errors are normally and identically distributed, a parametric test can be used. Analysis of Variance (ANOVA) is often the preferred approach, although one can also use a <em>t</em>-test to test whether the slope is significantly different from zero. If errors are not normally and identically distributed, then a randomization test should be used.</p>
<section id="the-coefficient-of-determination-r2" class="level3">
<h3 class="anchored" data-anchor-id="the-coefficient-of-determination-r2">The Coefficient of Determination <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"></h3>
<p>The most popular goodness-of-fit measure for linear regression is <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D">, a metric that represents the percentage of the variance in <img src="https://latex.codecogs.com/png.latex?y"> explained by our features <img src="https://latex.codecogs.com/png.latex?x">. More specifically, <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"> measures the percentage of variance explained normalized against the baseline variance of our model (which is just the variance of the mean): <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D=1-%5Cfrac%7B%5Csum_%7Bi=1%7D%5E%7Bn%7D(y_%7Bi%7D-%5Chat%7By%7D_%7Bi%7D)%5E%7B2%7D%7D%7B%5Csum_%7Bi=1%7D%5E%7Bn%7D(y_%7Bi%7D-%5Cbar%7By%7D_%7Bi%7D)%5E%7B2%7D%7D"></p>
<p>The highest possible value for <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"> is <strong>1</strong>, representing a model that captures 100% of the variance. A lower <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"> means that our model is doing worse (capturing less variance) of our data would.</p>
<p>R-squared is a statistical measure that represents the proportion of the variance in the dependent variable that is explained by the independent variables in the model. An R-squared value of 1 indicates that the model explains all the variance in the dependent variable, and a value of 0 indicates that the model explains none of the variances.</p>
</section>
<section id="adjusted-r2" class="level3">
<h3 class="anchored" data-anchor-id="adjusted-r2">Adjusted <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"></h3>
<p>Often the <strong>adjusted coefficient of determination</strong>, <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D_%7Badj%7D">, is quoted instead. The adjustment takes account of the sample size and the number of explanatory variables. With simple linear regression (only one explanatory variable) the adjustment <strong>only becomes noticeable for small sample sizes</strong> <img src="https://latex.codecogs.com/png.latex?(n%3C20)">. <img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D_%7Badj%7D=1-(1-R%5E%7B2%7D)%5Cfrac%7Bn-1%7D%7Bn-1-k%7D"> where:</p>
<ul>
<li><img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D"> is the unadjusted coefficient of determination</li>
<li><img src="https://latex.codecogs.com/png.latex?n"> is the number of bivariate observations, and</li>
<li><img src="https://latex.codecogs.com/png.latex?k"> is the number of explanatory variables <img src="https://latex.codecogs.com/png.latex?x_%7B1%7D,%20x_%7B2%7D,%20%5Cldots%20x_%7Bi%7D"> in our model.</li>
</ul>
<p>It is a better indicator of the model’s goodness of fit when comparing models with different numbers of independent variables.</p>
</section>
<section id="root-mean-squared-error-rmse" class="level3">
<h3 class="anchored" data-anchor-id="root-mean-squared-error-rmse">Root Mean Squared Error (RMSE)</h3>
<p>RMSE measures the difference between the predicted values and the actual values. A lower RMSE indicates a better fit of the model to the data.</p>
</section>
<section id="mean-absolute-error-mae" class="level3">
<h3 class="anchored" data-anchor-id="mean-absolute-error-mae">Mean Absolute Error (MAE)</h3>
<p>MAE measures the average difference between the predicted values and the actual values. A lower MAE indicates a better fit of the model to the data.</p>
<!--
### The Analysis of Variance

The ANalysis Of VAriance (ANOVA) splits an observed aggregate variability found inside a data set into two parts: _systematic factors_ and _random factors_. The **systematic factors** have a statistical influence on the given data set, while the **random factors** do not. The ANOVA test is usually performed to determine the influence that independent variables have on the dependent variable in a regression study.

$$
SS_{Reg}=\hat{\beta}_{1}^{2}\left(\displaystyle\sum_{i=1}^{n}x_{i}^{2}-\dfrac{\bigg(\displaystyle\sum_{i=1}^{n}x_{i}\bigg)^{2}}{n}\right)$$

$$
SS_{Total}=\displaystyle\sum_{i=1}^{n}y_{i}^{2}-\dfrac{1}{n}\Bigg(\displaystyle\sum_{i=1}^{n}y_{i}\Bigg)^{2}
$$

$$SSN_{Error}=SS_{Total}-SS_{Reg}$$




| Source of Variation | d.o.f. | Sum of Squared | Mean Squared | $F$-ratio | $p$-value |
|:-------:|:-----:|:------:|:------:|:------:|:------:|
| Regression | $1$ | $SS_{Reg}$  |  |   |   |
| Error | $n-2$  | $SS_{Error}$  |  |   |    |
| Total | $n-1$   | $SS_{Total}$  |  |   |    |

--->


<!-- -->

</section>
</section>

 ]]></description>
  <category>linear regression</category>
  <category>correlation</category>
  <category>goodness of fit</category>
  <guid>https://carobarrera.com/blog/2023/05/regression/</guid>
  <pubDate>Fri, 22 Dec 2023 23:00:00 GMT</pubDate>
  <media:content url="https://carobarrera.com/blog/2023/05/regression/network.png" medium="image" type="image/png" height="78" width="144"/>
</item>
<item>
  <title>Forecasting with XGBoost: An implementation in R</title>
  <dc:creator>José Caro</dc:creator>
  <link>https://carobarrera.com/blog/2021/08/XGBoost/</link>
  <description><![CDATA[ 





<p><img src="https://carobarrera.com/blog/2021/08/XGBoost/dplyr.png" alt="dplyr_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/dials.png" alt="dial_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/parsnip.png" alt="parsnip_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/recipes.png" alt="recipes_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/rsample.png" alt="rsample_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/tune.png" alt="tune_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/workflows.png" alt="workflows_sticker" style="height: 75px; width:75px;"> <img src="https://carobarrera.com/blog/2021/08/XGBoost/yardstick.png" alt="yardstick_sticker" style="height: 75px; width:75px;"></p>
<section id="introduction-to-the-xgboost-algorithm" class="level1">
<h1>1. Introduction to the XGBoost algorithm</h1>
<p>XGBoost, (<em>Extreme Gradient Boosting</em>), is one of the most widely used supervised <em>machine learning</em> algorithms that uses the <em>boosting</em> principle.</p>
<p>Supervised learning is one that has input variables <img src="https://latex.codecogs.com/png.latex?(x)"> and an output variable and an output variable <img src="https://latex.codecogs.com/png.latex?(Y)"> and uses an algorithm to learn the mapping function of the input <img src="https://latex.codecogs.com/png.latex?Y=f(X)"> to the output.The objective is to approximate the mapping function as accurately as possible so that when you have new input data <img src="https://latex.codecogs.com/png.latex?(x)">, the output variables <img src="https://latex.codecogs.com/png.latex?(Y)"> can be predicted for that data.</p>
<p>It is called supervised learning because the process of an algorithm learning from the training data set can be thought of as a teacher supervising the learning process. We know the correct answers, the algorithm iteratively makes predictions on the training data and is corrected by the teacher. Learning stops when the algorithm reaches an acceptable level of performance.</p>
<p>Supervised learning problems can be grouped into regression and classification problems.</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cbullet"> <strong>Classification:</strong> A classification problem is when the output variable is a category, such as “red” or “blue” or “disease” and “no disease”.</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cbullet"> <strong>Regression:</strong> A regression problem is when the output variable is a real value, such as “euros” or “kilograms”. Some common types of problems built on classification and regression include recommendation and time series prediction, respectively. Some popular examples of supervised machine learning algorithms are:</p>
<p><strong><em>Linear regression</em></strong> for regression problems.</p>
<p><strong><em>Random forest</em></strong> for classification and regression problems.</p>
<p><strong><em>Support vector machine</em></strong> for classification problems.</p>
<p>The <em>xgboost</em> algorithm is similar to <em>gradient boosting</em> but more efficient. It has linear model solutions and tree learning algorithms and is at least 10 times faster than existing implementations of <em>gradient boosting</em>. It supports several objective functions, including regression, classification and ranking. and what makes it fast is its ability to do parallel computations on a single machine.</p>
<p>In terms of efficiency, accuracy and feasibility it is more powerful than the <em>random forest</em> algorithm, for example or a neural network and since it has very high predictive power but relatively slow implementation, <em>xgboost</em> becomes a suitable choice for solving most regression, classification and ranking problems as well as user-created objective functions. It also has additional features to do cross validation and find the most important variables, which makes it interesting for <strong>credit scoring models</strong>, <strong>insurance claims</strong>, or where there are many parameters to be controlled to optimize the model.</p>
<p>As can be seen in the following graph (1), the XGBoost model has the best combination of prediction performance and processing time compared to other algorithms.</p>
<p><a href="performance.png"><img src="https://carobarrera.com/blog/2021/08/XGBoost/performance.png" class="img-fluid"></a></p>
<p>Chollet and Allaire, (2018) (2) summarize the value of XGBoost as follows:</p>
<blockquote class="blockquote">
<p>“<em>XGBoost</em> is used for problems where there is the availability of structured data is very wide, while <em>deep learning</em> is used for perceptual problems such as image classification. Users of the former almost always use the XGBoost library.”</p>
</blockquote>
<blockquote class="blockquote">
<p>“These are the two techniques that one should be most familiar with to be successful in applied machine learning today: gradient boosting machines, for shallow learning problems; and deep learning, for perceptual problems. In technical terms, this means that one should be familiar with XGBoost and Keras.”</p>
</blockquote>
<blockquote class="blockquote">
<footer>
– Francoise Chollet and J.J. Allaire
</footer>
</blockquote>
<p>Originally, XGBoost is a library written in C++ and exported to <strong>R</strong> in the <code>xgboost</code> package in our case, the XGBoost model has been trained using the package in <strong>R</strong> <code>caret</code>(3).</p>
</section>
<section id="objectives-and-data-preparation" class="level1">
<h1>2. Objectives and data preparation</h1>
<p>In this post and preliminary preview of the project with data application to the Spanish real estate market we train and tune a model based on the XGBoost algorithm using the <em>tidymodels</em> library of <code>R</code>. For this purpose and as data source we use the <a href="https://cran.r-project.org/web/packages/AmesHousing/AmesHousing.pdf">AmesHousing dataset</a> (4) which contains data of 82 variables for 2,930 properties in Ames County, Iowa. Our model will attempt to predict the selling price of the home.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb1" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb1-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Data load</span></span>
<span id="cb1-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(AmesHousing)</span>
<span id="cb1-3"></span>
<span id="cb1-4"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Cleaning and data preparation libraries</span></span>
<span id="cb1-5"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(janitor)</span>
<span id="cb1-6"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(dplyr)</span>
<span id="cb1-7"></span>
<span id="cb1-8"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Libraries needed</span></span>
<span id="cb1-9"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(rsample)</span>
<span id="cb1-10"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(recipes)</span>
<span id="cb1-11"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(parsnip)</span>
<span id="cb1-12"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(tune)</span>
<span id="cb1-13"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(dials)</span>
<span id="cb1-14"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(workflows)</span>
<span id="cb1-15"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(yardstick)</span>
<span id="cb1-16"></span>
<span id="cb1-17"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># acceleration of calculations with parallel processing (optional but useful)</span></span>
<span id="cb1-18"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(doParallel)</span>
<span id="cb1-19">all_cores <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> parallel<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">detectCores</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">logical =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span>)</span>
<span id="cb1-20"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">registerDoParallel</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">cores =</span> all_cores)</span></code></pre></div></div>
</div>
</div>
<section id="data-load-for-modelling" class="level3">
<h3 class="anchored" data-anchor-id="data-load-for-modelling">2.1. Data load for modelling</h3>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb2" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb2-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># random seed set.seed() for results to be replicable</span></span>
<span id="cb2-2"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># </span></span>
<span id="cb2-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">set.seed</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1234</span>)</span>
<span id="cb2-4"></span>
<span id="cb2-5"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># data load and cleaning names</span></span>
<span id="cb2-6">ames_data <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">make_ames</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb2-7">  janitor<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">clean_names</span>()</span></code></pre></div></div>
</div>
</div>
</section>
</section>
<section id="xgboost-process" class="level1">
<h1>3. XGBoost Process</h1>
<section id="exploratory-data-analysis." class="level2">
<h2 class="anchored" data-anchor-id="exploratory-data-analysis.">3.1. Exploratory Data Analysis.</h2>
<p>At this initial point, we would do summaries of the data and some simple graphs to get as detailed an understanding of the data as possible. For simplicity, we are going to skip this step but, in a real-world analysis, understanding the business issues and doing an effective EDA are often the crucial aspects that require the most time and analysis.</p>
</section>
<section id="splitting-the-data." class="level2">
<h2 class="anchored" data-anchor-id="splitting-the-data.">3.2. Splitting the Data.</h2>
<p>We now divide the data into training and test data. The training data is used for model training and hyperparameter fitting. Once trained, the model can be evaluated against the test data to assess accuracy. Typically, 80% of the data is used for model training, while simulation or testing is done with the remaining 20%.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb3-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Data split for training and testing. Stratification by sales price</span></span>
<span id="cb3-2">ames_split <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> rsample<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">initial_split</span>(</span>
<span id="cb3-3">  ames_data, </span>
<span id="cb3-4">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">prop =</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.8</span>, </span>
<span id="cb3-5">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">strata =</span> sale_price</span>
<span id="cb3-6">)</span></code></pre></div></div>
</div>
</div>
</section>
<section id="pre-processing." class="level2">
<h2 class="anchored" data-anchor-id="pre-processing.">3.3. Pre-processing.</h2>
<p>Preprocessing alters the data to make our model more predictive and the training process requires less computational calculations. Many models require careful and extensive variable preprocessing to produce accurate predictions. However, XGBoost is more robust to highly asymmetric and/or correlated data, so the amount of preprocessing required with XGBoost is minimal. However, we can still make use of some preprocessing, and in <strong>R</strong>, with the libraries that <code>tidymodels</code> uses, we use the <code>recipes</code> package to define these aforementioned preprocessing steps:</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb4" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb4-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># preprocessing routine</span></span>
<span id="cb4-2">preprocessing_recipe <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb4-3">  recipes<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">recipe</span>(sale_price <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> ., <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">data =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">training</span>(ames_split)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb4-4">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># categorical variables to factor variables conversion</span></span>
<span id="cb4-5">  recipes<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">step_string2factor</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">all_nominal</span>()) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb4-6">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># low frequency factor levels combination</span></span>
<span id="cb4-7">  recipes<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">step_other</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">all_nominal</span>(), <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">threshold =</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.01</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb4-8">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># remove of predictors without variance that do not provide predicitive information</span></span>
<span id="cb4-9">  recipes<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">step_nzv</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">all_nominal</span>()) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb4-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">prep</span>()</span></code></pre></div></div>
</div>
</div>
<p>As can be seen in the graph below, for the <code>neighborhood</code> variable, several of the factor levels with the smallest number of observations (less than 1% of the total number of observations) have been grouped into an <code>other</code> factor level. We did this preprocessing with the <code>step_other()</code> command in the previous section.</p>
<p><a href="others.png"><img src="https://carobarrera.com/blog/2021/08/XGBoost/others.png" class="img-fluid"></a></p>
</section>
<section id="cross-validation-split." class="level2">
<h2 class="anchored" data-anchor-id="cross-validation-split.">3.4. Cross-validation split.</h2>
<p>We apply the preprocessing previously defined with the <code>bake()</code> command. Then, we use cross-validation to randomly split the training data into additional training and test sets. We will use these additional cross-validation folds to adjust our hyperparameters in a later step.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb5" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb5-1">ames_cv_folds <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb5-2">  recipes<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">bake</span>(</span>
<span id="cb5-3">    preprocessing_recipe, </span>
<span id="cb5-4">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">new_data =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">training</span>(ames_split)</span>
<span id="cb5-5">  ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span>  </span>
<span id="cb5-6">  rsample<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">vfold_cv</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">v =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>)</span></code></pre></div></div>
</div>
</div>
</section>
<section id="xgboost-model-specification." class="level2">
<h2 class="anchored" data-anchor-id="xgboost-model-specification.">3.5. XGBoost model specification.</h2>
<p>We use the <code>parsnip</code> package to define the XGBoost model specification. We then use <code>boost_tree()</code> together with <code>tune()</code> to define the hyperparameters for tuning in a later step.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb6" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb6-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Model specification</span></span>
<span id="cb6-2">xgboost_model <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb6-3">  parsnip<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">boost_tree</span>(</span>
<span id="cb6-4">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">mode =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"regression"</span>,</span>
<span id="cb6-5">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">trees =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1000</span>,</span>
<span id="cb6-6">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">min_n =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tune</span>(),</span>
<span id="cb6-7">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">tree_depth =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tune</span>(),</span>
<span id="cb6-8">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">learn_rate =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tune</span>(),</span>
<span id="cb6-9">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">loss_reduction =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tune</span>()</span>
<span id="cb6-10">  ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb6-11">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">set_engine</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"xgboost"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">objective =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"reg:squarederror"</span>)</span></code></pre></div></div>
</div>
</div>
</section>
<section id="grid-specification." class="level2">
<h2 class="anchored" data-anchor-id="grid-specification.">3.6. Grid specification.</h2>
<p>Next, we use the <code>dials</code> package to specify the set of parameters.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb7-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># Grid setup</span></span>
<span id="cb7-2">xgboost_params <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb7-3">  dials<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">parameters</span>(</span>
<span id="cb7-4">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">min_n</span>(),</span>
<span id="cb7-5">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tree_depth</span>(),</span>
<span id="cb7-6">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">learn_rate</span>(),</span>
<span id="cb7-7">    <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">loss_reduction</span>()</span>
<span id="cb7-8">  )</span></code></pre></div></div>
</div>
</div>
<p>Then we set up the <em>grid</em> space. The <code>dials::grid_*</code> functions support several methods to define this space.Using the <code>dials::grid_max_entropy()</code> function covers the hyperparameter space so that any part of the space has an observed combination that does not lie too far from it.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb8-1">xgboost_grid <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb8-2">  dials<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">grid_max_entropy</span>(</span>
<span id="cb8-3">    xgboost_params, </span>
<span id="cb8-4">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">60</span></span>
<span id="cb8-5">  )</span>
<span id="cb8-6"></span>
<span id="cb8-7">knitr<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">kable</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">head</span>(xgboost_grid))</span></code></pre></div></div>
</div>
</div>
<table class="caption-top table">
<thead>
<tr class="header">
<th style="text-align: center;">min_n</th>
<th style="text-align: center;">tree_depth</th>
<th style="text-align: center;">learn_rate</th>
<th style="text-align: center;">loss_reduction</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">34</td>
<td style="text-align: center;">1</td>
<td style="text-align: center;">0.0118682</td>
<td style="text-align: center;">29.9649253</td>
</tr>
<tr class="even">
<td style="text-align: center;">38</td>
<td style="text-align: center;">12</td>
<td style="text-align: center;">0.0001291</td>
<td style="text-align: center;">0.6156496</td>
</tr>
<tr class="odd">
<td style="text-align: center;">6</td>
<td style="text-align: center;">7</td>
<td style="text-align: center;">0.0000949</td>
<td style="text-align: center;">0.0000000</td>
</tr>
<tr class="even">
<td style="text-align: center;">32</td>
<td style="text-align: center;">4</td>
<td style="text-align: center;">0.0000005</td>
<td style="text-align: center;">0.0000367</td>
</tr>
<tr class="odd">
<td style="text-align: center;">14</td>
<td style="text-align: center;">2</td>
<td style="text-align: center;">0.0001833</td>
<td style="text-align: center;">0.0000000</td>
</tr>
<tr class="even">
<td style="text-align: center;">31</td>
<td style="text-align: center;">8</td>
<td style="text-align: center;">0.0000000</td>
<td style="text-align: center;">1.4345098</td>
</tr>
</tbody>
</table>
<p>To fit our model, we perform a <em>grid</em> search over that <code>xgboost_grid</code> grid space to identify the hyperparameter values that have the lowest prediction error.</p>
</section>
<section id="workflow-setup" class="level2">
<h2 class="anchored" data-anchor-id="workflow-setup">3.7. Workflow setup</h2>
<p>We use the new <code>tidymodel</code> workflow package to add a formula to our XGBoost model specification.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb9-1">xgboost_wf <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb9-2">  workflows<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">workflow</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb9-3">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">add_model</span>(xgboost_model) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span> </span>
<span id="cb9-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">add_formula</span>(sale_price <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> .)</span></code></pre></div></div>
</div>
</div>
</section>
<section id="model-fitting" class="level2">
<h2 class="anchored" data-anchor-id="model-fitting">3.8. Model fitting</h2>
<p>The “fitting” is where the <code>tidymodels</code> package <em>ecosystem</em> actually comes into play. Here’s a quick breakdown of the objects passed to the first 4 arguments of our <code>tune_grid()</code> call below:</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cbullet"> “object”: <code>xgboost_wf</code>, which is a workflow we defined by the <code>parsnip</code> and <code>workflows</code> packages.</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cbullet"> “resamples”: <code>ames_cv_folds</code> as defined by the <code>rsample</code> and <code>recipes</code> packages.</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cbullet"> “grid”: <code>xgboost_grid</code> our space as defined by the <code>dials</code> package.</p>
<p><img src="https://latex.codecogs.com/png.latex?%5Cbullet"> “metrics”: the <code>yardstick</code> package defines the set of metrics used to evaluate model performance.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb10-1"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># hyperparameters tuning</span></span>
<span id="cb10-2">xgboost_tuned <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> tune<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">tune_grid</span>(</span>
<span id="cb10-3">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">object =</span> xgboost_wf,</span>
<span id="cb10-4">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">resamples =</span> ames_cv_folds,</span>
<span id="cb10-5">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">grid =</span> xgboost_grid,</span>
<span id="cb10-6">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">metrics =</span> yardstick<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">metric_set</span>(rmse, rsq, mae),</span>
<span id="cb10-7">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">control =</span> tune<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">control_grid</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">verbose =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>)</span>
<span id="cb10-8">)</span></code></pre></div></div>
</div>
</div>
<p>In the above code block, <code>tune_grid()</code> performed a search for the <em>grid</em> on all 60 parameter combinations defined with <code>xgboost_grid</code> and used 5-fold cross-validation along with the <em>rmse</em> (root mean square error), <em>rsq</em> (<img src="https://latex.codecogs.com/png.latex?R%5E%7B2%7D">) and <em>mae</em> (mean absolute error) to measure prediction accuracy. Therefore, our fit only fits (worth the redundancy) <img src="https://latex.codecogs.com/png.latex?60%5Ctimes%205=300"> XGBoost models, each with 1,000 trees, all in search of the optimal hyperparameters. The computation was considerably time consuming and not without its share of problems. The hyperparameter values that performed best in minimizing the mean square error are shown below:</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb11" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb11-1">xgboost_tuned <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb11-2">  tune<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">show_best</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">metric =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"rmse"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb11-3">  knitr<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">kable</span>()</span></code></pre></div></div>
</div>
</div>
<table class="caption-top table">
<colgroup>
<col style="width: 7%">
<col style="width: 13%">
<col style="width: 13%">
<col style="width: 17%">
<col style="width: 9%">
<col style="width: 13%">
<col style="width: 10%">
<col style="width: 3%">
<col style="width: 10%">
</colgroup>
<thead>
<tr class="header">
<th style="text-align: center;">min_n</th>
<th style="text-align: center;">tree_depth</th>
<th style="text-align: center;">learn_rate</th>
<th style="text-align: center;">loss_reduction</th>
<th style="text-align: center;">.metric</th>
<th style="text-align: center;">.estimator</th>
<th style="text-align: center;">mean</th>
<th style="text-align: center;">n</th>
<th style="text-align: center;">std_err</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">12</td>
<td style="text-align: center;">7</td>
<td style="text-align: center;">0.0346875</td>
<td style="text-align: center;">0.0451186</td>
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">25561.99</td>
<td style="text-align: center;">5</td>
<td style="text-align: center;">2983.927</td>
</tr>
<tr class="even">
<td style="text-align: center;">9</td>
<td style="text-align: center;">13</td>
<td style="text-align: center;">0.0183617</td>
<td style="text-align: center;">0.1042750</td>
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">25576.99</td>
<td style="text-align: center;">5</td>
<td style="text-align: center;">2687.849</td>
</tr>
<tr class="odd">
<td style="text-align: center;">23</td>
<td style="text-align: center;">5</td>
<td style="text-align: center;">0.0788798</td>
<td style="text-align: center;">0.7513677</td>
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">25645.38</td>
<td style="text-align: center;">5</td>
<td style="text-align: center;">2461.057</td>
</tr>
<tr class="even">
<td style="text-align: center;">11</td>
<td style="text-align: center;">6</td>
<td style="text-align: center;">0.0091690</td>
<td style="text-align: center;">0.0000001</td>
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">25669.84</td>
<td style="text-align: center;">5</td>
<td style="text-align: center;">2706.457</td>
</tr>
<tr class="odd">
<td style="text-align: center;">10</td>
<td style="text-align: center;">2</td>
<td style="text-align: center;">0.0108475</td>
<td style="text-align: center;">0.0000003</td>
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">25883.23</td>
<td style="text-align: center;">5</td>
<td style="text-align: center;">2828.172</td>
</tr>
</tbody>
</table>
<p>Next, we isolate the best performing hyperparameter values.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb12" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb12-1">xgboost_best_params <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> xgboost_tuned <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb12-2">  tune<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">select_best</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"rmse"</span>)</span>
<span id="cb12-3"></span>
<span id="cb12-4">knitr<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">kable</span>(xgboost_best_params)</span></code></pre></div></div>
</div>
</div>
<table class="caption-top table">
<thead>
<tr class="header">
<th style="text-align: center;">min_n</th>
<th style="text-align: center;">tree_depth</th>
<th style="text-align: center;">learn_rate</th>
<th style="text-align: center;">loss_reduction</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">12</td>
<td style="text-align: center;">7</td>
<td style="text-align: center;">0.0346875</td>
<td style="text-align: center;">0.0451186</td>
</tr>
</tbody>
</table>
<p>We finish the XGBoost model using the best parameter we have set.</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb13" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb13-1">xgboost_model_final <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> xgboost_model <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span> </span>
<span id="cb13-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">finalize_model</span>(xgboost_best_params)</span></code></pre></div></div>
</div>
</div>
</section>
<section id="performance-evaluation-with-test-data." class="level2">
<h2 class="anchored" data-anchor-id="performance-evaluation-with-test-data.">3.9. Performance Evaluation with Test Data.</h2>
<p>Now that we have trained our model, we need to evaluate the performance of the model. We use the test data from step 1 (that 20% of data that was not used in model training) to evaluate the performance.</p>
<p>We use the <em>rmse</em> (Root Mean Squared Error), rsq (R Squared) and mae (Mean Absolute Value) metrics from the <code>yardstick</code> package in our model evaluation.</p>
<p>First, we evaluated the training data metrics:</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb14" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb14-1">train_processed <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">bake</span>(preprocessing_recipe,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">new_data =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">training</span>(ames_split))</span>
<span id="cb14-2"></span>
<span id="cb14-3">train_prediction <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> xgboost_model_final <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb14-4">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># model fit on all training data</span></span>
<span id="cb14-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fit</span>(</span>
<span id="cb14-6">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">formula =</span> sale_price <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> ., </span>
<span id="cb14-7">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">data    =</span> train_processed</span>
<span id="cb14-8">  ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb14-9">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># prediction of sales prices on training data</span></span>
<span id="cb14-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">predict</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">new_data =</span> train_processed) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb14-11">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">bind_cols</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">training</span>(ames_split))</span>
<span id="cb14-12"></span>
<span id="cb14-13">xgboost_score_train <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb14-14">  train_prediction <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb14-15">  yardstick<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">metrics</span>(sale_price, .pred) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb14-16">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">.estimate =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">format</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">round</span>(.estimate, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>), <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">big.mark =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">","</span>))</span>
<span id="cb14-17"></span>
<span id="cb14-18">knitr<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">kable</span>(xgboost_score_train)</span></code></pre></div></div>
</div>
</div>
<table class="caption-top table">
<thead>
<tr class="header">
<th style="text-align: center;">.metric</th>
<th style="text-align: center;">estimator.</th>
<th style="text-align: center;">.estimate</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">3,807.24</td>
</tr>
<tr class="even">
<td style="text-align: center;">rsq</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">1.00</td>
</tr>
<tr class="odd">
<td style="text-align: center;">mae</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">2,747.17</td>
</tr>
</tbody>
</table>
<p>And now for the rest of the data:</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb15" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb15-1">test_processed  <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">bake</span>(preprocessing_recipe, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">new_data =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">testing</span>(ames_split))</span>
<span id="cb15-2"></span>
<span id="cb15-3">test_prediction <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> xgboost_model_final <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb15-4">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># model fit on all training data</span></span>
<span id="cb15-5">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">fit</span>(</span>
<span id="cb15-6">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">formula =</span> sale_price <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">~</span> ., </span>
<span id="cb15-7">    <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">data    =</span> train_processed</span>
<span id="cb15-8">  ) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb15-9">  <span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># use of the training fit model for test data prediction</span></span>
<span id="cb15-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">predict</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">new_data =</span> test_processed) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb15-11">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">bind_cols</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">testing</span>(ames_split))</span>
<span id="cb15-12"></span>
<span id="cb15-13"><span class="co" style="color: #5E5E5E;
background-color: null;
font-style: inherit;"># measuring the accuracy of our model using `yardstick`.</span></span>
<span id="cb15-14">xgboost_score <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> </span>
<span id="cb15-15">  test_prediction <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb15-16">  yardstick<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">metrics</span>(sale_price, .pred) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb15-17">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">.estimate =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">format</span>(<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">round</span>(.estimate, <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>), <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">big.mark =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">","</span>))</span>
<span id="cb15-18"></span>
<span id="cb15-19">knitr<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">kable</span>(xgboost_score)</span></code></pre></div></div>
</div>
</div>
<table class="caption-top table">
<thead>
<tr class="header">
<th style="text-align: center;">.metric</th>
<th style="text-align: center;">estimator.</th>
<th style="text-align: center;">.estimate</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: center;">rmse</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">30,217.58</td>
</tr>
<tr class="even">
<td style="text-align: center;">rsq</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">0.87</td>
</tr>
<tr class="odd">
<td style="text-align: center;">mae</td>
<td style="text-align: center;">standard</td>
<td style="text-align: center;">15,728.22</td>
</tr>
</tbody>
</table>
<p>The above metrics in the test data are significantly worse than the metrics in our training data, so we know that there is some overfitting in our model. This highlights the importance of using test data, rather than training data, to evaluate model performance.</p>
<p>To quickly check that there is not a problem with our model predictions, we can obtain the plot of the test data residuals:</p>
<div class="fold s">
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb16" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb16-1">house_prediction_residual <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> test_prediction <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb16-2">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">arrange</span>(.pred) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb16-3">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">mutate</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">residual_pct =</span> (sale_price <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span> .pred) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">/</span> .pred) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span></span>
<span id="cb16-4">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">select</span>(.pred, residual_pct)</span>
<span id="cb16-5"></span>
<span id="cb16-6"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ggplot</span>(house_prediction_residual, <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">aes</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">x =</span> .pred, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">y =</span> residual_pct)) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb16-7">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">geom_point</span>() <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb16-8">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">xlab</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Predicted Sale Price"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb16-9">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">ylab</span>(<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Residual (%)"</span>) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb16-10">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_x_continuous</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labels =</span> scales<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">dollar_format</span>()) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">+</span></span>
<span id="cb16-11">  <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">scale_y_continuous</span>(<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labels =</span> scales<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">::</span>percent)</span></code></pre></div></div>
</div>
</div>
<p><a href="residuals.png"><img src="https://carobarrera.com/blog/2021/08/XGBoost/residuals.png" class="img-fluid"></a></p>
<p>The above plot does not show very obvious trends in residuals. This indicates that, at a very high level, our model is not systematically making inaccurate predictions for homes with certain predicted sales prices. We would do more model validation here for a real-world analysis, but, for the sake of this publication, the above graph is good enough for our purpose.</p>
</section>
</section>
<section id="conclusions" class="level1">
<h1>Conclusions</h1>
<p>The objective of this analysis was to work through the training process of an XGBoost model in <strong>R</strong> using the <code>tidymodels</code> package, and to learn the basics of how the algorithm works, although we have not put too much emphasis on the performance of our model, the foundations for future lines of research with this tool have been laid.</p>
<p>We have seen that the <code>tidymodels</code> library provides us with a standard process and tools to handle resampling (<code>rsample</code>), data preprocessing (<code>recipes</code>), model specification (<code>parsnip</code>), fitting (<code>tune</code>) and model validation (<code>yardstick</code>). In this sense, the <code>tidymodels</code> ability to <code>sort</code> the machine learning process is a step-change improvement for machine learning accessibility in <strong>R</strong>; thus, it is easier to train and understand the XGBoost model training process.</p>
</section>
<section id="references" class="level1">
<h1>References</h1>
<ol type="1">
<li><p>https://towardsdatascience.com/https-medium-com-vishalmorde-xgboost-algorithm-long-she-may-rein-edd9f99be63d</p></li>
<li><p>Chollet, F. and Allaire, J. J. (2018): <em>“Deep Learning with R,”</em> Ed. Manning Publications.</p></li>
<li><p>Kuhn, M. (2008). <em>“Building Predictive Models in R Using the caret Package,”</em> Journal of Statistical Software, 28(5), 1-26; doi:http://dx.doi.org/10.18637/jss.v028.i05.</p></li>
<li><p>De Cock, D. (2011). <em>“Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project,”</em>: Journal of Statistics Education, Volume 19, Number 3.</p></li>
<li><p>Chen, T. and Guestrin, C. (2016): <em>“XGBoost: A Scalable Tree Boosting System,”</em> doi:10.1145/2939672.2939785</p></li>
<li><p>Friedman, J.H. (2001): <em>“Greedy Function Approximation: A Gradient Boosting Machine,”</em> Annals of Statistics, pp.&nbsp;1189–1232.</p></li>
</ol>


<!-- -->

</section>

 ]]></description>
  <category>XGBoost</category>
  <category>data analysis</category>
  <category>forecasting</category>
  <category>machine learning</category>
  <guid>https://carobarrera.com/blog/2021/08/XGBoost/</guid>
  <pubDate>Mon, 07 Nov 2022 23:00:00 GMT</pubDate>
</item>
<item>
  <title>Scientific Maps Analysis with the R package bibliometrix</title>
  <dc:creator>José Caro</dc:creator>
  <link>https://carobarrera.com/blog/2022/09/bibliometric/</link>
  <description><![CDATA[ 





<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/preregistered_large_color.png" class="img-fluid" alt="Preregistered">   <img src="https://carobarrera.com/blog/2022/09/bibliometric/data_large_color.png" class="img-fluid" alt="Open data">   <img src="https://carobarrera.com/blog/2022/09/bibliometric/materials_large_color.png" class="img-fluid" alt="Open"></p>
<section id="introduction" class="level1">
<h1>Introduction</h1>
</section>
<section id="bibliographic-data" class="level1">
<h1>Bibliographic Data</h1>
<p>The bibliographic data are based on the search query that has been made. For the moment we have focused on one of the lines of the <a href="../../../../now/index.html">SEJ670</a> UCO Research Group, <strong>neophilia/neophobia in tourism</strong>. Needles to say that it would be applied to other research lines; however, watch out when filtering and redefining the database because we have to remove areas that are not of our interest, for example, in our selection there were references about zoology, biology, etc…, given that there is also neophilia and neophobia, especially the latter, specifically in animals, well, we must be careful because some area can sneak in the form of journals and references not related to our research area.</p>
<p><strong>Data source</strong>: Clarivate Analytics Web of Science (<a href="http://apps.webofknowledge.com" class="uri">http://apps.webofknowledge.com</a>)</p>
<p><strong>Format</strong>: Bibtex</p>
<p><strong>Query</strong>: “Web of Science Core Collection”</p>
<p><strong>Range</strong>: 1995-2022</p>
<p><strong>Document type</strong>: All</p>
<p><strong>Query date</strong>: September, 2022</p>
</section>
<section id="package-installation-data-loading-and-conversion" class="level1">
<h1>Package installation, data loading and conversion</h1>
<p>As the objective is to see what analysis the tool does and how it does it, the steps of installation, data loading and conversion have been omitted, although it is an important step since it must be done in <code>R</code>, after downloading the data from WoS or Scopus, which also has its own technique. After a lot of research we managed to find a way to merge both databases and it works perfectly, and it also automatically omits duplicate references. <strong>With this procedure we can have both databases in a single file and although WoS has more references (as a general rule, although not always), Scopus almost always has some different one/s and even if they are few, at least we practically cover the two most important sources of articles.</strong> It is worth mentioning that in the last updates of this <code>Bibliometrix</code> package other open bibliographic databases such as <em>OpenAlex</em>, <em>Dimensions</em>, <em>Lens</em>, and from the medical field such as <em>PubMed</em> and <em>Cochrane</em> have been included.</p>
</section>
<section id="descriptive-analysis" class="level1">
<h1>1. Descriptive Analysis</h1>
<p>The descriptive analysis of the package provides a lot of information on the annual development of research, the most productive <img src="https://latex.codecogs.com/png.latex?k"> authors, articles, countries and relevant keywords.</p>
<section id="main-findings-on-the-database-analysed" class="level2">
<h2 class="anchored" data-anchor-id="main-findings-on-the-database-analysed">1.1. Main findings on the database analysed</h2>
<p>The following table shows a summary of the data and other interesting classifications: <em>n_umber of authors</em>, <em>number of documents</em>, <em>scientific production for each year</em> (and its average growth), the <em>most productive authors</em>, the <em>countries</em> and the <em>corresponding citations</em>, the <em>most relevant journals</em>, the <em>most relevant keywords</em>, etc…</p>
<p>Moreover, this information can also be obtained in graphs.</p>
<div class="cell">
<div class="cell-output cell-output-stdout">
<pre><code>

MAIN INFORMATION ABOUT DATA

 Timespan                              1995 : 2022 
 Sources (Journals, Books, etc)        69 
 Documents                             152 
 Annual Growth Rate %                  9.64 
 Document Average Age                  9.52 
 Average citations per doc             29.24 
 Average citations per year per doc    2.425 
 References                            7867 
 
DOCUMENT TYPES                     
 article                         130 
 article; book chapter           2 
 article; early access           5 
 article; proceedings paper      1 
 editorial material              4 
 letter                          1 
 proceedings paper               3 
 review                          6 
 
DOCUMENT CONTENTS
 Keywords Plus (ID)                    617 
 Author's Keywords (DE)                512 
 
AUTHORS
 Authors                               467 
 Author Appearances                    531 
 Authors of single-authored docs       18 
 
AUTHORS COLLABORATION
 Single-authored docs                  19 
 Documents per Author                  0.325 
 Co-Authors per Doc                    3.49 
 International co-authorships %        36.18 
 

Annual Scientific Production

 Year    Articles
    1995        1
    2000        2
    2003        2
    2005        3
    2006        2
    2007        3
    2008        5
    2009        3
    2010        7
    2011        2
    2012        6
    2013        5
    2014        8
    2015       11
    2016       11
    2017       13
    2018       13
    2019       11
    2020       17
    2021       10
    2022       12

Annual Percentage Growth Rate 9.64 


Most Productive Authors

     Authors        Articles   Authors        Articles Fractionalized
1  METTKE-HOFMANN C        8 METTKE-HOFMANN C                    3.53
2  BUGNYAR T               4 STHAPIT E                           1.33
3  EVES A                  4 EVES A                              1.20
4  MORAND-FERRON J         4 MORAND-FERRON J                     1.20
5  BISAZZA A               3 [ANONYMOUS] A                       1.00
6  CARACCIOLO F            3 AKYUZ BG                            1.00
7  GRIFFIN AS              3 ANTONAKIS J                         1.00
8  KIM YG                  3 ARMELAGOS GJ                        1.00
9  VERNEAU F               3 BERTI I                             1.00
10 WIDDIG A                3 CHAKRABARTI B                       1.00


Top manuscripts per citations

                                 Paper                                      DOI  TC TCperYear  NTC
1  KIM YG, 2009, INT J HOSP MANAG               10.1016/j.ijhm.2008.11.005      346     20.35 2.38
2  CHANG RCY, 2010, ANN TOURIS RES              10.1016/j.annals.2010.03.007    276     17.25 3.21
3  ANTONAKIS J, 2017, LEADERSH Q                10.1016/j.leaqua.2017.01.006    192     21.33 4.79
4  VERBEKE W, 2005, BR FOOD J                   10.1108/00070700510629779       183      8.71 1.97
5  KIM YG, 2010, INT J HOSP MANAG               10.1016/j.ijhm.2009.10.015      178     11.12 2.07
6  CAVIGELLI SA, 2003, PROC NATL ACAD SCI U S A 10.1073/pnas.2535721100         161      7.00 1.22
7  HUGHES RN, 2007, NEUROSCI BIOBEHAV REV       10.1016/j.neubiorev.2006.11.004 110      5.79 2.84
8  MORAND-FERRON J, 2011, BEHAV ECOL            10.1093/beheco/arr120           106      7.07 1.49
9  DAY RL, 2003, ANIM BEHAV                     10.1006/anbe.2003.2074          103      4.48 0.78
10 REVERDY C, 2008, APPETITE                    10.1016/j.appet.2008.01.010     101      5.61 1.51


Corresponding Author's Countries

          Country Articles   Freq SCP MCP MCP_Ratio
1  UNITED KINGDOM       24 0.1622  17   7     0.292
2  USA                  18 0.1216  12   6     0.333
3  CHINA                14 0.0946   5   9     0.643
4  AUSTRALIA            13 0.0878   9   4     0.308
5  GERMANY               9 0.0608   4   5     0.556
6  CANADA                8 0.0541   5   3     0.375
7  ITALY                 7 0.0473   7   0     0.000
8  AUSTRIA               6 0.0405   2   4     0.667
9  BELGIUM               3 0.0203   1   2     0.667
10 BRAZIL                3 0.0203   2   1     0.333


SCP: Single Country Publications

MCP: Multiple Country Publications


Total Citations per Country

      Country      Total Citations Average Article Citations
1  UNITED KINGDOM             1122                     46.75
2  USA                         639                     35.50
3  CHINA                       490                     35.00
4  AUSTRALIA                   275                     21.15
5  BELGIUM                     193                     64.33
6  SWITZERLAND                 192                     96.00
7  AUSTRIA                     187                     31.17
8  GERMANY                     158                     17.56
9  ARGENTINA                   143                     71.50
10 ITALY                       143                     20.43


Most Relevant Sources

                                    Sources        Articles
1  ANIMAL BEHAVIOUR                                      17
2  ETHOLOGY                                               8
3  ANIMAL COGNITION                                       7
4  FOOD QUALITY AND PREFERENCE                            7
5  BEHAVIORAL ECOLOGY                                     6
6  BRITISH FOOD JOURNAL                                   6
7  SCIENTIFIC REPORTS                                     6
8  APPETITE                                               5
9  INTERNATIONAL JOURNAL OF HOSPITALITY MANAGEMENT        5
10 PLOS ONE                                               5


Most Relevant Keywords

   Author Keywords (DE)      Articles Keywords-Plus (ID)     Articles
1             NEOPHILIA            24           NEOPHOBIA          48
2             FOOD NEOPHOBIA       23           BEHAVIOR           29
3             NEOPHOBIA            22           NEOPHILIA          21
4             EXPLORATION          13           ATTITUDES          17
5             FOOD                 11           SCALE              17
6             FOOD TOURISM         11           SATISFACTION       16
7             FOOD NEOPHILIA       10           TOURISM            15
8             PERSONALITY           9           EVOLUTION          13
9             INNOVATION            8           INFORMATION        13
10            LOCAL FOOD            8           MODEL              13</code></pre>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Descriptive Analysis-1.png" class="img-fluid figure-img" width="672"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Descriptive Analysis-2.png" class="img-fluid figure-img" width="672"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Descriptive Analysis-3.png" class="img-fluid figure-img" width="672"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Descriptive Analysis-4.png" class="img-fluid figure-img" width="672"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Descriptive Analysis-5.png" class="img-fluid figure-img" width="672"></p>
</figure>
</div>
</div>
</div>
</section>
<section id="most-cited-references" class="level2">
<h2 class="anchored" data-anchor-id="most-cited-references">1.2. Most cited references</h2>
<p>We can also obtain the most cited references and their number:</p>
<div class="cell">
<div class="cell-output cell-output-stdout">
<pre><code>                                                                                   [,1]
PLINER P, 1992, APPETITE, V19, P105, DOI 10.1016/0195-6663(92)90014-W                55
GREENBERG R, 2001, CURR ORNITHOL, V16, P119                                          32
COHEN E, 2004, ANN TOURISM RES, V31, P755, DOI 10.1016/J.ANNALS.2004.02.003          29
RITCHEY PN, 2003, APPETITE, V40, P163, DOI 10.1016/S0195-6663(02)00134-4             27
METTKE-HOFMANN C, 2002, ETHOLOGY, V108, P249, DOI 10.1046/J.1439-0310.2002.00773.X   24
TUORILA H, 2001, FOOD QUAL PREFER, V12, P29, DOI 10.1016/S0950-3293(00)00025-2       24
KIM YG, 2009, INT J HOSP MANAG, V28, P423, DOI 10.1016/J.IJHM.2008.11.005            21
KIM YG, 2010, INT J HOSP MANAG, V29, P216, DOI 10.1016/J.IJHM.2009.10.015            19
QUAN S, 2004, TOURISM MANAGE, V25, P297, DOI 10.1016/S0261-5177(03)00130-4           19
REALE D, 2007, BIOL REV, V82, P291, DOI 10.1111/J.1469-185X.2007.00010.X             19
CHANG RCY, 2010, ANN TOURISM RES, V37, P989, DOI 10.1016/J.ANNALS.2010.03.007        16
PLINER P, 2006, FRONT NUTR SCI, P75, DOI 10.1079/9780851990323.0075                  16
FORNELL C, 1981, J MARKETING RES, V18, P39, DOI 10.2307/3151312                      15
JI MJ, 2016, TOURISM MANAGE, V57, P387, DOI 10.1016/J.TOURMAN.2016.06.003            15
MAK AHN, 2012, INT J HOSP MANAG, V31, P928, DOI 10.1016/J.IJHM.2011.10.012           15
CHANG RCY, 2011, TOURISM MANAGE, V32, P307, DOI 10.1016/J.TOURMAN.2010.02.009        14
FISCHLER C, 1988, SOC SCI INFORM, V27, P275, DOI 10.1177/053901888027002005          14
GREENBERG RUSSELL, 2003, P175                                                        14
MARTIN LB, 2005, BEHAV ECOL, V16, P702, DOI 10.1093/BEHECO/ARI044                    14
SOL D, 2011, PLOS ONE, V6, DOI 10.1371/JOURNAL.PONE.0019535                          14</code></pre>
</div>
</div>
</section>
</section>
<section id="co-citation-analysis-structure" class="level1">
<h1>2: Co-citation analysis structure</h1>
<p>Citation analysis is another remarkable tool of the bibliometric analysis offered by the package. It shows the structure of a specific field through the links between nodes (e.g.&nbsp;authors, articles, journal). The interesting option is that the edges can be interpreted differently depending on the type of network, i.e.&nbsp;co-citations, direct citations, bibliographic linking, etc…. This is very useful and can be exploited quite a lot.</p>
<p>Below we have taken the three standard examples shown in the original reference but adapted them to our database.</p>
<p><em>First</em>, a co-citation network showing the relationships between the cited-referred works (nodes).</p>
<p><em>Second</em>, a co-citation network that uses the cited journals as the unit of analysis.</p>
<p>The dimensions useful for commenting on co-citation networks are: <strong>(i)</strong> centrality and peripherality of nodes, <strong>(ii)</strong> their proximity and distance, <strong>(iii)</strong> strength of links, <strong>(iv)</strong> clusters, <strong>(v)</strong> bridging contributions.</p>
<p><em>Third</em>, a historiography that is built on direct quotations. It traces the intellectual links in a historical order. The cited works of thousands of authors contained in a collection of published scientific articles are sufficient to reconstruct the historiographical structure of the field, pointing out the basic works in it.</p>
<section id="analysis-of-co-citations-of-articles-references" class="level2">
<h2 class="anchored" data-anchor-id="analysis-of-co-citations-of-articles-references">Analysis of co-citations of articles (references)</h2>
<p>This is the typical <em>VosViewer</em> graph, in this case for a visualization of the co-cite network. The graph parameters have not been modified to make it clearer but the <strong>graph options</strong> shown below are mainly for visual fine tuning. Without seeing the code this may be useless but it is interesting to know that it can be tweaked and made more readable. The interesting thing would be to be able to eliminate those co-citations that appear isolated and focus only on those where there are relationships.</p>
<ul>
<li><p><code>n = 50</code> (this function traces the top 50 cited references)</p></li>
<li><p><code>type = "fruchterman"</code> (the network layout is generated by the <a href="https://github.com/gephi/gephi/wiki/Fruchterman-Reingold">Fruchterman-Reingold algorithm</a>, there is the option of other types of algorithms, although I have not tested them and I do not know if they would work for us)</p></li>
<li><p><code>size.cex = TRUE</code> (the size of the vertices is proportional to its degree)</p></li>
<li><p><code>size = 20</code> (maximum size of the vertices)</p></li>
<li><p><code>remove.multiple = FALSE</code> (multiple edges are not eliminated, the opposite is <code>TRUE</code>)</p></li>
<li><p><code>labelsize = 1</code> (defines the size of the vertices labels)</p></li>
<li><p><code>edgesize = 10</code> (the thickness of the edges is proportional to their strength. <code>Edgesize</code> defines the maximum value of the thickness)</p></li>
<li><p><code>edges.min = 5</code> (only traces edges with a force greater than or equal to 5)</p></li>
<li><p>all other arguments assume default values.</p></li>
</ul>
<div class="cell">
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co-citation network-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
</section>
<section id="analysis-of-journal-co-citations-source" class="level2">
<h2 class="anchored" data-anchor-id="analysis-of-journal-co-citations-source">Analysis of journal co-citations (source)</h2>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb3" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb3-1">wos<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">metaTagExtraction</span>(wos,<span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"CR_SO"</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb3-2">NetMatrix <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">biblioNetwork</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">analysis =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"co-citation"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">network =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"sources"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb3-3">net<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">networkPlot</span>(NetMatrix, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">Title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Co-Citation Network"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">type =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"auto"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size.cex=</span><span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">15</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">remove.multiple=</span><span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labelsize=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edgesize =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edges.min=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co-citation source network-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
<p>Descriptive analysis of the characteristics of the journal citation network.</p>
<p>This analysis is similar to the previous one but with journal citations.</p>
<div class="cell">
<div class="cell-output cell-output-stdout">
<pre><code>

Main statistics about the network

 Size                                  2876 
 Density                               0.034 
 Transitivity                          0.364 
 Diameter                              4 
 Degree Centralization                 0.481 
 Average path length                   2.284 
 </code></pre>
</div>
</div>
</section>
</section>
<section id="historiography---direct-citation-links" class="level1">
<h1>3: Historiography - Direct citation links</h1>
<p>This analysis provides us with direct citation links by author and year, which is interesting to see who cites whom and when.</p>
<div class="cell">
<div class="cell-output cell-output-stdout">
<pre><code>
WOS DB:
Searching local citations (LCS) by reference items (SR) and DOIs...

Analyzing 10475 reference items...

Found 49 documents with no empty Local Citations (LCS)</code></pre>
</div>
</div>
<div class="cell">
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Historiograph-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
<div class="cell-output cell-output-stdout">
<pre><code>
 Legend

                                                                       Label
1              HEINRICH B, 1995, ANIM BEHAV DOI 10.1016/0003-3472(95)80130-8
2                        DAY RL, 2003, ANIM BEHAV DOI 10.1006/ANBE.2003.2074
3           METTKE-HOFMANN C, 2005, BEHAVIOUR DOI 10.1163/156853905774539427
4               METTKE-HOFMANN C, 2005, BEHAV ECOL DOI 10.1093/BEHECO/ARH159
5         ECHEVERRIA AI, 2008, ETHOLOGY DOI 10.1111/J.1439-0310.2008.01512.X
6              KIM YG, 2009, INT J HOSP MANAG DOI 10.1016/J.IJHM.2008.11.005
7      METTKE-HOFMANN C, 2009, ETHOLOGY DOI 10.1111/J.1439-0310.2009.01632.X
8                  BERGMAN TJ, 2009, ANIM COGN DOI 10.1007/S10071-008-0171-2
9              KIM YG, 2010, INT J HOSP MANAG DOI 10.1016/J.IJHM.2009.10.015
10               MORAND-FERRON J, 2011, BEHAV ECOL DOI 10.1093/BEHECO/ARR120
11              RAUDENBUSH B, 2012, APPETITE DOI 10.1016/J.APPET.2012.02.042
12             KIM YG, 2013, INT J HOSP MANAG DOI 10.1016/J.IJHM.2012.06.005
13 DIMITROVSKI D, 2017, J TRAVEL TOUR MARK DOI 10.1080/10548408.2016.1193100
14      MAK AHN, 2017, ASIA PAC J TOUR RES DOI 10.1080/10941665.2016.1175488
15    GRIFFIN AS, 2017, CURR OPIN BEHAV SCI DOI 10.1016/J.COBEHA.2017.01.004
16          HSU FC, 2018, TOUR RECREAT RES DOI 10.1080/02508281.2018.1475879
17                AKYUZ BG, 2019, ANATOLIA DOI 10.1080/13032917.2019.1595072
18         BAAH NG, 2020, CURR ISSUES TOUR DOI 10.1080/13683500.2019.1619676
19                   LAI MY, 2020, J TRAVEL RES DOI 10.1177/0047287519867144
20         OKUMUS B, 2021, TOUR MANAG PERSPECT DOI 10.1016/J.TMP.2020.100773
                                                                                                                            Author_Keywords
1                                                                                                                                      &lt;NA&gt;
2                                                                                                                                      &lt;NA&gt;
3                                                                             CONTEXT SPECIFICITY; EXPLORATION; NEOPHILIA; NEOPHOBIA; BIRDS
4                                                              ENVIRONMENTAL ASSESSMENT; NEOPHILIA; NOMADS; PARROTS; PSITTACIDAE; RESIDENTS
5                                                                                                                                      &lt;NA&gt;
6                                                 LOCAL FOOD; FOOD CONSUMPTION; FOOD TOURISM; CULINARY TOURISM; MOTIVATION; GROUNDED THEORY
7                                                                                                                                      &lt;NA&gt;
8                        NOVEL OBJECTS; NEOPHILIA; EXPLORATION; PRIMATE; NEOPHOBIA; THEROPITHECUS; GELADA; PAPIO URSINUS; BABOON; COGNITION
9                                                                          FOOD INVOLVEMENT; FOOD NEOPHOBIA; LOCAL FOOD; FESTIVALS; LOYALTY
10                                                       AGE DIFFERENCES; INNOVATION; PERSONALITY; PROBLEM SOLVING; SOCIAL; LEARNING; STATE
11                                                                                        PSYCHOLOGY; FOOD NEOPHOBIA; PHYSIOLOGICAL AROUSAL
12                                                                       EMPIRICAL VERIFICATION; LOCAL FOOD; FOOD CONSUMPTION; FOOD TOURISM
13                       FOOD MARKETS; SATISFACTION; TOURIST ATTRACTION; MOTIVATIONS; FOOD MARKET; LOVERS; FOOD NEOPHILIA; BARCELONA; SPAIN
14 TOURIST FOOD CONSUMPTION; FOOD-RELATED PERSONALITY TRAITS; FOOD; NEOPHOBIA; VARIETY-SEEKING; TOURIST'S PARADOX; SENSORY-SPECIFIC SATIETY
15                                                                                                                                     &lt;NA&gt;
16                                                      FOOD TOURISM; FOOD NEOPHOBIA; SENSATION-SEEKING; TRADITIONAL FOOD CHOICE; BEHAVIOUR
17                                                  CULINARY TOURISM; GASTRONOMY TOURISM; LOCAL FOOD; TRAVEL MOTIVATIONS; TRAVEL INTENTIONS
18                                                               GASTRONOMY TOURISM; FOOD TOURISM; NEOPHOBIA; NEOPHILIC TENDENCY; ATTITUDES
19                              DESTINATION FOOD IMAGE; FOOD TOURISM; FOOD NEOPHOBIA; POTENTIAL CHINESE; TOURIST; FORMATIVE IMAGE CONSTRUCT
20                                        GENDER; GENERATION; FOOD NEOPHOBIA; FOOD NEOPHILIA; FOOD TOURISM; ETHNIC; FOOD; CHINESE TRAVELERS
                                                                                                                                                            KeywordsPlus
1                                                                                                                                                                   &lt;NA&gt;
2                       TAMARINS LEONTOPITHECUS-ROSALIA; PRIMATE COGNITION; FEEDING NEOPHOBIA; CALLITHRIX-KUHLI; TOOL USE; BEHAVIOR; EVOLUTION; PATTERNS; SAGUINUS; SIZE
3         GUPPY POECILIA-RETICULATA; INDIVIDUAL-DIFFERENCES; EXPLORATORY-BEHAVIOR; GREAT TITS; REALIZED HERITABILITY; RISK-TAKING; PARUS-MAJOR; ANIMALS; PIGS; NEOPHOBIA
4                                       NUCLEOTIDE-SEQUENCES; DECISION-MAKING; BIRD COMMUNITY; PHYLOGENY; INFORMATION; STRATEGIES; NEOPHOBIA; BEHAVIOR; STIMULUS; GUINEA
5                                                                                  HOUSE SPARROWS; GROUP-SIZE; NEOPHOBIA; EXPLORATION; RAVENS; NEOPHILIA; BEHAVIOR; SONG
6                                                                                                      TOURISM; CHOICE; PERCEPTION; SELECTION; PATTERNS; HERITAGE; TASTE
7                                                       REALIZED HERITABILITY; MEMORY; EXPLORATION; BEHAVIOR; INFORMATION; HIPPOCAMPUS; FAMILIARITY; SUCCESS; LIFE; CUES
8                                         BEHAVIORAL ECOLOGY; FEEDING NEOPHOBIA; GREAT TITS; TRADE-OFFS; EVOLUTION; MONKEYS; EXPLORATION; HABITAT; BIRDS; RESPONSIVENESS
9                                                         CONSUMER INVOLVEMENT; NEOPHOBIA; TOURISM; CHOICE; SCALE; FAMILIAR; UNFAMILIAR; ATTITUDES; VARIABLES; SELECTION
10                  TIT PARUS-MAJOR; BEHAVIORAL FLEXIBILITY; INDIVIDUAL VARIATION; FORAGING; INNOVATION; SOCIAL RANK; GREAT TITS; BODY-SIZE; NEOPHOBIA; NEOPHILIA; BIRDS
11                                                                                                                                                                  &lt;NA&gt;
12                                                                                                                                          SCALE DEVELOPMENT; NEOPHOBIA
13                                       STRUCTURAL MODEL; CULINARY TOURISM; FARMERS MARKETS; NEOPHOBIA; FESTIVAL; ATTITUDES; VISITORS; SCALE; SATISFACTION; DESTINATION
14                                                              HONG-KONG; NEOPHOBIA; SCALE; CHOICE; MODEL; ANTECEDENTS; INFORMATION; EXPERIENCES; ACCEPTANCE; ATTITUDES
15       INVASION SUCCESS; BEHAVIORAL FLEXIBILITY; REPRODUCTIVE OUTPUT; PERSONALITY-TRAITS; TROPHIC DYNAMICS; NEST PREDATION; RISK-TAKING; LAND-COVER; BRAIN SIZE; BIRDS
16                                 SENSATION SEEKING; PLANNED BEHAVIOR; LOCAL FOOD; PERSONALITY-TRAITS; EUROPEAN FOOD; TOURISM; ATTITUDES; CHOICE; NEOPHOBIA; ACCEPTANCE
17                                  CULINARY TOURISM; CONSUMER INVOLVEMENT; DESTINATION; SEGMENTATION; NEOPHOBIA; SCALE; NATIONALITY; INFORMATION; ATTRACTION; NUTRITION
18                                             PURCHASE INTENTIONS; CONSUMER ATTITUDES; NOVELTY-SEEKING; HONG-KONG; FOOD; BEHAVIOR; ANTECEDENTS; GASTRONOMY; CHOICE; FIT
19 PARTIAL LEAST-SQUARES; STRUCTURAL EQUATION MODELS; LOCAL FOOD; CULINARY; TOURISM; FORMATIVE MEASUREMENT; PSYCHOLOGICAL-FACTORS; PLS; SATISFACTION; INFORMATION; PLACE
20                                        EATING BEHAVIOR; NOVELTY-SEEKING; FAMILIAR; CHILDREN; HABITS; SATISFACTION; WILLINGNESS; RESEMBLANCE; PERSONALITY; INVOLVEMENT
                                DOI Year LCS GCS
1      10.1016/0003-3472(95)80130-8 1995  10  71
2            10.1006/anbe.2003.2074 2003   8 103
3        10.1163/156853905774539427 2005   7  54
4             10.1093/beheco/arh159 2005   7  42
5  10.1111/j.1439-0310.2008.01512.x 2008   6  44
6        10.1016/j.ijhm.2008.11.005 2009  21 346
7  10.1111/j.1439-0310.2009.01632.x 2009   5  33
8         10.1007/s10071-008-0171-2 2009   5  58
9        10.1016/j.ijhm.2009.10.015 2010  19 178
10            10.1093/beheco/arr120 2011   5 106
11      10.1016/j.appet.2012.02.042 2012   5  41
12       10.1016/j.ijhm.2012.06.005 2013   8  85
13    10.1080/10548408.2016.1193100 2017   8  40
14    10.1080/10941665.2016.1175488 2017  13  73
15     10.1016/j.cobeha.2017.01.004 2017   6  51
16    10.1080/02508281.2018.1475879 2018   6  21
17    10.1080/13032917.2019.1595072 2019   4  10
18    10.1080/13683500.2019.1619676 2020   5  13
19         10.1177/0047287519867144 2020   6  34
20        10.1016/j.tmp.2020.100773 2021   5  11</code></pre>
</div>
</div>
</section>
<section id="conceptual-structure---co-word-analysis" class="level1">
<h1>4: Conceptual structure - Co-word analysis</h1>
<p>Co-Word Networks show the conceptual structure, which uncovers the links between concepts through co-occurrences of terms.</p>
<p>The conceptual structure can be used to understand the topics being discussed (research front) and to identify which are the most important and most recent topics.</p>
<p>The tool divides the whole time span into different periods and compares the conceptual structures which is useful to analyze the evolution of the topics over time.</p>
<p>The package is able to analyze keywords, but also terms in article titles and abstracts. It does this by means of network analysis or correspondence analysis (CA) or multiple correspondence analysis (MCA). CA and MCA visualize the conceptual structure in a two-dimensional graph, which I also show below.</p>
<section id="joint-word-analysis-using-keyword-co-occurrences" class="level2">
<h2 class="anchored" data-anchor-id="joint-word-analysis-using-keyword-co-occurrences">Joint word analysis using keyword co-occurrences</h2>
<p>As with the previous graph, the <strong>options</strong> are to make the graph more readable:</p>
<ul>
<li><p><code>normalize = "association"</code> (the vertex similarities are normalized using association strength)</p></li>
<li><p><code>n = 50</code> (the function traces the top 50 cited references)</p></li>
<li><p><code>type = "fruchterman"</code> (The network trace is generated using the <em>Fruchterman-Reingold</em> algorithm)</p></li>
<li><p><code>size.cex = TRUE</code> (the size of the vertices is proportional to their degree)</p></li>
<li><p><code>size = 20</code> (maximum size of the vertices)</p></li>
<li><p><code>remove.multiple = FALSE</code> (multiple edges are not removed)</p></li>
<li><p><code>labelsize = 3</code> (defines the maximum size of the vertices labels)</p></li>
<li><p><code>label.cex = TRUE</code> (size of vertices labels is proportional to their degree)</p></li>
<li><p><code>edgesize = 10</code> (thickness of the edges is proportional to their degree. <code>Edgesize</code> defines the maximum value of the thickness)</p></li>
<li><p><code>label.n = 30</code> (labels are drawn only for the top 30 vertices)</p></li>
<li><p><code>edges.min = 25</code> (Only plot edges with a strength greater than or equal to 25)</p></li>
<li><p>all other arguments assume default values.</p></li>
</ul>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb7" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb7-1">NetMatrix <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">biblioNetwork</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">analysis =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"co-occurrences"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">network =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"keywords"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb7-2">net<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">networkPlot</span>(NetMatrix, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">normalize=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"association"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">Title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Keyword Co-occurrences"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">type =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"fruchterman"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size.cex=</span><span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">20</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">remove.multiple=</span>F, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edgesize =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labelsize=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">label.cex=</span><span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">label.n=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">30</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edges.min=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">2</span>)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Keyword co-occurrences-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
</section>
<section id="joint-word-analysis-by-means-of-correspondence-analysis" class="level2">
<h2 class="anchored" data-anchor-id="joint-word-analysis-by-means-of-correspondence-analysis">Joint word analysis by means of correspondence analysis</h2>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb8" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb8-1"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">suppressWarnings</span>(</span>
<span id="cb8-2">CS <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">conceptualStructure</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">method=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"MCA"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">field=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ID"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">minDegree=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">15</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">clust=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">stemming=</span><span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labelsize=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">15</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">documents=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">20</span>)</span>
<span id="cb8-3">)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co-word Analysis-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co-word Analysis-2.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co-word Analysis-3.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co-word Analysis-4.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
</section>
</section>
<section id="thematic-maps" class="level1">
<h1>5: Thematic maps</h1>
<p>The co-word analysis draws clusters of the keywords. Themes are considered, whose density and centrality can be used to classify the themes and draw a two-dimensional diagram.</p>
<p>The thematic map is a very intuitive diagram and we can analyze the topics according to the quadrant in which they are placed: <em>(1)</em> upper right quadrant: <em>main topics</em> (as qualified by the authors of the tool); <em>(2)</em> lower right quadrant: <strong>basic topics</strong>; <em>(3)</em> lower left quadrant: <strong>emerging or disappearing topics</strong>; <em>(4)</em> upper left quadrant: <strong>very specialized/niche topics</strong>.</p>
<p>El análisis de co-palabras dibuja clusters de las palabras clave. Se consideran temas, cuya densidad y centralidad pueden utilizarse para clasificar los temas y trazar un diagrama bidimensional.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb9" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb9-1">Map<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">thematicMap</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">field =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"ID"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">250</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">minfreq =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>,</span>
<span id="cb9-2">  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">stemming =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">FALSE</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size =</span> <span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.7</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n.labels=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">repel =</span> <span class="cn" style="color: #8f5902;
background-color: null;
font-style: inherit;">TRUE</span>)</span>
<span id="cb9-3"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">plot</span>(Map<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>map)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/ThematicMap-1.png" class="img-fluid figure-img" width="864"></p>
</figure>
</div>
</div>
</div>
<p>The description of the cluster can then be requested from the program:</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb10" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb10-1">Clusters<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span>Map<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>words[<span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">order</span>(Map<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>words<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>Cluster,<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">-</span>Map<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>words<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>Occurrences),]</span>
<span id="cb10-2"><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">library</span>(dplyr)</span></code></pre></div></div>
<div class="cell-output cell-output-stderr">
<pre><code>
Attaching package: 'dplyr'</code></pre>
</div>
<div class="cell-output cell-output-stderr">
<pre><code>The following objects are masked from 'package:stats':

    filter, lag</code></pre>
</div>
<div class="cell-output cell-output-stderr">
<pre><code>The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union</code></pre>
</div>
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb14" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb14-1">CL <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> Clusters <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">group_by</span>(.data<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>Cluster_Label) <span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">%&gt;%</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">top_n</span>(<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">5</span>, .data<span class="sc" style="color: #5E5E5E;
background-color: null;
font-style: inherit;">$</span>Occurrences)</span>
<span id="cb14-2">CL</span></code></pre></div></div>
<div class="cell-output cell-output-stdout">
<pre><code># A tibble: 24 × 9
# Groups:   Cluster_Label [4]
   Occurrences Words           Cluster Color    Cluster_Label Cluster_Frequency btw_centrality clos_centrality pagerank_centrality
         &lt;dbl&gt; &lt;chr&gt;             &lt;dbl&gt; &lt;chr&gt;    &lt;chr&gt;                     &lt;dbl&gt;          &lt;dbl&gt;           &lt;dbl&gt;               &lt;dbl&gt;
 1           9 willingness           1 #E41A1C… willingness                 106           579.         0.00193             0.0112 
 2           7 acceptance            1 #E41A1C… willingness                 106           565.         0.00189             0.00872
 3           6 familiar              1 #E41A1C… willingness                 106           159.         0.00176             0.00660
 4           5 eating behavior       1 #E41A1C… willingness                 106           334.         0.00180             0.00572
 5           5 food neophobia        1 #E41A1C… willingness                 106           226.         0.00181             0.00480
 6          48 neophobia             2 #377EB8… neophobia                   416          4804.         0.00242             0.0438 
 7          29 behavior              2 #377EB8… neophobia                   416          4508.         0.00239             0.0271 
 8          21 neophilia             2 #377EB8… neophobia                   416          1163.         0.00205             0.0219 
 9          13 evolution             2 #377EB8… neophobia                   416          1001.         0.00204             0.0130 
10          12 exploration           2 #377EB8… neophobia                   416           936.         0.00204             0.0136 
# ℹ 14 more rows</code></pre>
</div>
</div>
</section>
<section id="social-structure---collaboration-analysis" class="level1">
<h1>6: Social structure - Collaboration analysis</h1>
<p>This last section is also interesting. Collaborative networks show how authors, institutions (e.g., universities or departments) and countries relate to others in the field we are analyzing, in this case neophilia/neophobia in tourism. For example, the first figure below is a “Co-author network”. It uncovers regular study groups, hidden groups of scholars and key authors. The second figure is called an “Educational Collaboration Network” and uncovers relevant institutions in a specific research field and their relationships.</p>
<section id="author-collaboration-network" class="level2">
<h2 class="anchored" data-anchor-id="author-collaboration-network">Author collaboration network</h2>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb16" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb16-1">NetMatrix <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">biblioNetwork</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">analysis =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"collaboration"</span>,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">network =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"authors"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb16-2">net<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">networkPlot</span>(NetMatrix,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">Title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Author collaboration"</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">type =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"auto"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size.cex=</span>T,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edgesize =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labelsize=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Au collaboration network-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
</section>
<section id="educational-collaboration-network" class="level2">
<h2 class="anchored" data-anchor-id="educational-collaboration-network">Educational collaboration network</h2>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb17" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb17-1">NetMatrix <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">biblioNetwork</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">analysis =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"collaboration"</span>,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">network =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"universities"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb17-2">net<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">networkPlot</span>(NetMatrix,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">50</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">Title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Edu collaboration"</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">type =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"auto"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">4</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size.cex=</span>F,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edgesize =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">3</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labelsize=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Edu collaboration network-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
</section>
<section id="collaboration-network-between-countries" class="level2">
<h2 class="anchored" data-anchor-id="collaboration-network-between-countries">Collaboration network between countries</h2>
<p>Finally, we can also obtain a graph of the collaboration network between countries, which can be useful to see between which countries there is more collaboration on the topic in question. For example, it can be seen that the country that has the most relationships is the United Kingdom. Spain, on the other hand, establishes collaborations with Japan, South Africa, Indonesia, Germany and China.</p>
<div class="cell">
<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb18" style="background: #f1f3f5;"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb18-1">wos <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">metaTagExtraction</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">Field =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"AU_CO"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb18-2">NetMatrix <span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">&lt;-</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">biblioNetwork</span>(wos, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">analysis =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"collaboration"</span>,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">network =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"countries"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">sep =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">";"</span>)</span>
<span id="cb18-3">net<span class="ot" style="color: #003B4F;
background-color: null;
font-style: inherit;">=</span><span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">networkPlot</span>(NetMatrix,  <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">n =</span> <span class="fu" style="color: #4758AB;
background-color: null;
font-style: inherit;">dim</span>(NetMatrix)[<span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>], <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">Title =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"Country collaboration"</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">type =</span> <span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"circle"</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size=</span><span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">10</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">size.cex=</span>T,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">edgesize =</span> <span class="dv" style="color: #AD0000;
background-color: null;
font-style: inherit;">1</span>,<span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">labelsize=</span><span class="fl" style="color: #AD0000;
background-color: null;
font-style: inherit;">0.6</span>, <span class="at" style="color: #657422;
background-color: null;
font-style: inherit;">cluster=</span><span class="st" style="color: #20794D;
background-color: null;
font-style: inherit;">"none"</span>)</span></code></pre></div></div>
<div class="cell-output-display">
<div>
<figure class="figure">
<p><img src="https://carobarrera.com/blog/2022/09/bibliometric/index_files/figure-html/Co collaboration network-1.png" class="img-fluid figure-img" width="960"></p>
</figure>
</div>
</div>
</div>
</section>
</section>
<section id="final-comments" class="level1">
<h1>Final comments</h1>
<p>Apart from this library that works in <code>R</code>, the authors have created an application that does the same without programming, and there is even an additional tool, also useful. The only thing you need is to have the DB in the correct format (merged WoS and Scopus, if you want, etc…) because otherwise, it does not read it but if you do it right, the result is the same as we have shown here but without programming <code>R</code> practically (only to transform the files).</p>
</section>
<section id="references" class="level1">
<h1>References</h1>
<p>This work is done with the <strong>R</strong> package <code>bibliometrix</code> and on the basis proposed by Aria and Cuccurullo, adapted and modified for our research line.</p>
<p>Aria, M., &amp; Cuccurullo, C. (2017). <strong>bibliometrix: An R-tool for comprehensive science mapping analysis</strong>, <em>Journal of Informetrics</em>, 11(4), pp 959-9753643, (<a href="https://www.bibliometrix.org" class="uri">https://www.bibliometrix.org</a>).</p>


<!-- -->

</section>

<div id="quarto-appendix" class="default"><section class="quarto-appendix-contents" id="quarto-citation"><h2 class="anchored quarto-appendix-heading">Citation</h2><div><div class="quarto-appendix-secondary-label">BibTeX citation:</div><pre class="sourceCode code-with-copy quarto-appendix-bibtex"><code class="sourceCode bibtex">@online{caro2022,
  author = {Caro, José},
  title = {Scientific {Maps} {Analysis} with the {`R`} Package
    `Bibliometrix`},
  date = {2022-09-09},
  url = {https://carobarrera.com/blog/2022/09/bibliometric/},
  doi = {10.5281/zenodo.10079724},
  langid = {en}
}
</code></pre><div class="quarto-appendix-secondary-label">For attribution, please cite this work as:</div><div id="ref-caro2022" class="csl-entry quarto-appendix-citeas">
Caro, José. 2022. <span>“Scientific Maps Analysis with the `R` Package
`Bibliometrix`.”</span> September 9, 2022. <a href="https://doi.org/10.5281/zenodo.10079724">https://doi.org/10.5281/zenodo.10079724</a>.
</div></div></section></div> ]]></description>
  <category>bibliometry</category>
  <category>mapping</category>
  <category>bibliometric analysis</category>
  <guid>https://carobarrera.com/blog/2022/09/bibliometric/</guid>
  <pubDate>Thu, 08 Sep 2022 22:00:00 GMT</pubDate>
  <media:content url="https://carobarrera.com/blog/2022/09/bibliometric/graph.png" medium="image" type="image/png" height="139" width="144"/>
</item>
<item>
  <title>Blog</title>
  <dc:creator>José Caro</dc:creator>
  <link>https://carobarrera.com/blog/</link>
  <description><![CDATA[ 





<div class="cell">
<style type="text/css">
#title-block-header .description {
    display: none;
}
</style>
</div>
<p>When I’m not blogging about <a href="../demography/">demography</a>, I blog here about economics, finance, data science, computing or any other interesting topic for the community. It’s a way to keep my ideas organized and gathered in one place rather than scattered across countless directories and folders on my computer. This makes them easily accessible—not just to me, but to anyone interested—though please remember that these posts reflect only my opinions, methods…</p>
<section id="section" class="level2">
<h2 class="anchored" data-anchor-id="section">2026</h2>
<div id="listing-posts_2026" class="quarto-listing quarto-listing-container-custom">
<div class="list blog">
    
    <div class="blog-entry" data-index="0" data-categories="aGVhdG1hcCUyQ2RhdGElMjB2aXolMkNydW5uaW5n" data-listing-date-sort="1790632800000" data-listing-file-modified-sort="1788613304809" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="6" data-listing-word-count-sort="1112">
        <div class="metadata">          
            <p class="date listing-date">September 29, 2026</p>
        </div>
        <div class="body">
            <p class="title listing-title"><a href="../blog/2026/09/index.html">Mapping the Miles: A Running Log as a Calendar Heatmap</a></p>

            
            <div class="post-categories listing-categories">
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('heatmap'); return false;">
                    heatmap
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('data viz'); return false;">
                    data viz
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('running'); return false;">
                    running
                </div>
                
            </div>
            

            <p class="description listing-description">Running heatmap</p>
            
            
        </div>
        
        <div class="thumbnail">
            <img src="https://carobarrera.com/blog/2026/09/heatmap.png" alt="Mapping the Miles: A Running Log as a Calendar Heatmap" title="Mapping the Miles: A Running Log as a Calendar Heatmap">
        </div>
        
    </div>
    
</div>
<div class="listing-no-matching d-none">No matching items</div>
</div>
<!-- ## 2025

:::{#posts_2025}
:::

## 2024

:::{#posts_2024}
:::-->
</section>
<section id="section-1" class="level2">
<h2 class="anchored" data-anchor-id="section-1">2023</h2>
<div id="listing-posts_2023" class="quarto-listing quarto-listing-container-custom">
<div class="list blog">
    
    <div class="blog-entry" data-index="0" data-categories="bGluZWFyJTIwcmVncmVzc2lvbiUyQ2NvcnJlbGF0aW9uJTJDZ29vZG5lc3MlMjBvZiUyMGZpdA==" data-listing-date-sort="1703286000000" data-listing-file-modified-sort="1788613304791" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="11" data-listing-word-count-sort="2069">
        <div class="metadata">          
            <p class="date listing-date">December 23, 2023</p>
        </div>
        <div class="body">
            <p class="title listing-title"><a href="../blog/2023/05/regression/index.html">Understanding Linear Regression</a></p>

            
            <div class="post-categories listing-categories">
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('linear regression'); return false;">
                    linear regression
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('correlation'); return false;">
                    correlation
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('goodness of fit'); return false;">
                    goodness of fit
                </div>
                
            </div>
            

            <p class="description listing-description">linear regression</p>
            
            
        </div>
        
        <div class="thumbnail">
            <img src="https://carobarrera.com/blog/2023/05/regression/network.png" alt="Understanding Linear Regression" title="Understanding Linear Regression">
        </div>
        
    </div>
    
</div>
<div class="listing-no-matching d-none">No matching items</div>
</div>
</section>
<section id="section-2" class="level2">
<h2 class="anchored" data-anchor-id="section-2">2022</h2>
<div id="listing-posts_2022" class="quarto-listing quarto-listing-container-custom">
<div class="list blog">
    
    <div class="blog-entry" data-index="0" data-categories="YmlibGlvbWV0cnklMkNtYXBwaW5nJTJDYmlibGlvbWV0cmljJTIwYW5hbHlzaXM=" data-listing-date-sort="1662674400000" data-listing-file-modified-sort="1788613304784" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="11" data-listing-word-count-sort="2118">
        <div class="metadata">          
            <p class="date listing-date">September 9, 2022</p>
        </div>
        <div class="body">
            <p class="title listing-title"><a href="../blog/2022/09/bibliometric/index.html">Scientific Maps Analysis with the `R` package `bibliometrix`</a></p>

            
            <div class="post-categories listing-categories">
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('bibliometry'); return false;">
                    bibliometry
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('mapping'); return false;">
                    mapping
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('bibliometric analysis'); return false;">
                    bibliometric analysis
                </div>
                
            </div>
            

            <p class="description listing-description">Example of scientific maps for bibliometric analysis</p>
            
            
            <p class="post-doi"><img src="https://carobarrera.com/files/doi.svg" class="doi-icon img-fluid" alt="DOI"> <a href="https://doi.org/10.5281/zenodo.10079724" target="_blank" rel="noopener noreferrer">10.5281/zenodo.10079724</a></p>
            
        </div>
        
        <div class="thumbnail">
            <img src="https://carobarrera.com/blog/2022/09/bibliometric/graph.png" alt="Scientific Maps Analysis with the `R` package `bibliometrix`" title="Scientific Maps Analysis with the `R` package `bibliometrix`">
        </div>
        
    </div>
    
</div>
<div class="listing-no-matching d-none">No matching items</div>
</div>
</section>
<section id="section-3" class="level2">
<h2 class="anchored" data-anchor-id="section-3">2021</h2>
<div id="listing-posts_2021" class="quarto-listing quarto-listing-container-custom">
<div class="list blog">
    
    <div class="blog-entry" data-index="0" data-categories="WEdCb29zdCUyQ2RhdGElMjBhbmFseXNpcyUyQ2ZvcmVjYXN0aW5nJTJDbWFjaGluZSUyMGxlYXJuaW5n" data-listing-date-sort="1667862000000" data-listing-file-modified-sort="1788613304735" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="15" data-listing-word-count-sort="2808">
        <div class="metadata">          
            <p class="date listing-date">November 8, 2022</p>
        </div>
        <div class="body">
            <p class="title listing-title"><a href="../blog/2021/08/XGBoost/index.html">Forecasting with XGBoost: An implementation in R</a></p>

            
            <div class="post-categories listing-categories">
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('XGBoost'); return false;">
                    XGBoost
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('data analysis'); return false;">
                    data analysis
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('forecasting'); return false;">
                    forecasting
                </div>
                
                <div class="post-category listing-category" onclick="window.quartoListingCategory('machine learning'); return false;">
                    machine learning
                </div>
                
            </div>
            

            <p class="description listing-description">Application in `R` of the XGBoost machine learning algorithm for predictive modelling</p>
            
            
        </div>
        
    </div>
    
</div>
<div class="listing-no-matching d-none">No matching items</div>
</div>
<!--
## 2020

:::{#posts_2020}
:::
-->



</section>

 ]]></description>
  <guid>https://carobarrera.com/blog/</guid>
  <pubDate>Sat, 05 Sep 2026 14:10:47 GMT</pubDate>
</item>
</channel>
</rss>
