Course Overview

Welcome to the course notes for Statistical Research Methodology (Course Code: DDS 6106), offered by the Department of Applied Statistics and Data Science at the Prasanna School of Public Health, Manipal Academy of Higher Education (MAHE).

Course Metadata:

  • Course Code: DDS 6106
  • Programme: M.Sc. Data Science, Biostatistics / Digital Epidemiology
  • Semester: Second Year, Semester 3
  • Credits: 4 Credits (\(L-T-P-C: 2-0-4-4\))
  • Prerequisites: All courses offered till Semester 3.
  • Synopsis: To provide the necessary foundation to simulate data and apply Markov Chain Monte Carlo (MCMC) techniques.

Course Outcomes (COs)

On successful completion of this course, students will be able to:

CO Description Bloom’s Level
CO 1 Outline the methodologies to carry out a research study C4
CO 2 Reproduce appropriate methods of simulation C1
CO 3 Demonstrate construction and analysis of simulation models C3
CO 4 Apply the techniques of simulation and MCMC C3
CO 5 Summarize how simulation and MCMC tools are used in industries to solve real-time problems C6
CO 6 Estimate and predict using MCMC C6

How this book is organized

Chapter Unit Hours Breakdown Key Topics
Chapter 1 Unit 1: Introduction to Research Methodology 07L + 05 SDL Research design, literature survey, protocol/thesis/manuscript (IMRAD), sampling designs
Chapter 2 Unit 2: Introduction to Simulation 05L + 02 SDL + 14 PRC LCG, Inverse-Transform, Accept-Reject, Poisson processes, EDF & influence functions
Chapter 3 Unit 3: Monte Carlo Methods 10L + 05 SDL + 10 PRC Variance reduction (control/antithetic variates, stratified, importance sampling), Bootstrap/Jackknife
Chapter 4 Unit 4: Markov Chain Monte Carlo 05L + 05 SDL + 10 PRC MCMC foundations, Metropolis-Hastings, Gibbs sampler, Ising/Potts, CFTP, Bayesian statistics
Chapter 5 Unit 5: Monte Carlo Optimization 03L + 03 SDL + 06 PRC Score function method, Robbins-Monro stochastic approximation, multi-level splitting, adaptive MCMC

Course Notation Guide

To assist students navigating statistical notation across different units:

  • Symbol \(\beta\):
    • Unit 2: Rate (or inverse-scale) parameter in Exponential and Gamma distributions.
    • Unit 4: Inverse temperature parameter (\(\beta = 1 / k_B T\)) in Ising and Potts models.
    • Appendix B: Vector of regression coefficients in Bayesian linear regression.
  • Symbol \(\alpha\):
    • Unit 2: Shape parameter of the Gamma distribution.
    • Unit 4: Probability of accepting a proposed candidate state in Metropolis-Hastings.
    • Unit 1: Significance level (\(\alpha\)) in statistical hypothesis testing.
  • Symbol \(\theta\):
    • Units 1-3, 5: General model parameter vector under estimation or inference.
    • Unit 5: Parameter value being optimized in Robbins-Monro root-finding.

Course Unit Learning Objectives

0.0.0.1 Unit 1: Introduction to Research Methodology (07 Lecture + 05 SDL = 12 Hours)

  • Describe Research Meaning, Objectives, & Types (C1 — Remember): Define scientific research and classify studies across descriptive, analytical, exploratory, hypothesis-testing, basic, applied, quantitative, and qualitative types.
  • Distinguish Research Methods vs. Research Methodology (C2 — Understand): Explain the fundamental distinction between data collection tools (methods) and the logical justification of the study design (methodology).
  • Examine the 11 Steps in the Research Process (C4 — Analysis): Deconstruct the 11 sequential, interdependent stages of scientific inquiry from problem formulation to report dissemination.
  • Recognize Research Ethics & Academic Integrity (C1 — Remember): Apply Belmont Report principles (Autonomy, Beneficence, Justice), navigate Institutional Review Board (IRB) reviews, and avoid scientific misconduct.
  • Master Literature Search & Referencing (C4 — Analysis): Construct Boolean search queries, create structured literature matrices, and utilize APA 7th, Vancouver, and BibTeX citation styles.
  • Evaluate Scientific Formats & Reporting Guidelines (C2 — Understand): Contrast SPIRIT Research Protocols, 5 Academic Thesis Archetypes, and IMRaD Journal Manuscripts with CONSORT 2010, STROBE, and PRISMA 2020 checklists.
  • Discuss & Select Research Designs (C2 — Understand): Compare observational designs (Cross-Sectional, Case-Control, Cohort) and experimental designs (RCTs, RBD, Latin Square, Factorial).
  • Calculate Sampling Designs & Sample Sizes (C3 — Apply): Compute probability sample sizes (\(n\)), derive Stratified Neyman and Cost-Constrained Optimum Allocations, and calculate Design Effect (\(DEFF\)).

0.0.0.2 Unit 2: Introduction to Simulation (05 Lecture + 02 SDL + 14 Practicals = 21 Hours)

  • Outline Foundations of Simulation (C1 — Remember): Explain pseudo-random numbers, random variables, and random vectors within a formal computational framework.
  • Derive & Implement Simulation Algorithms (C3 — Apply): Construct Linear Congruential Generators (LCG), Inverse-Transform, Accept-Reject, Box-Muller, and Cholesky vector algorithms step by step.
  • Simulate Stochastic Processes (C3 — Apply): Generate Homogeneous & Non-Homogeneous Poisson Processes, Markov Chains, and Jump Processes.
  • Generate Random Permutations (C3 — Apply): Implement the Fisher-Yates shuffle algorithm to generate unbiased random permutations.
  • Analyze Empirical Distributions & Influence Functions (C5 — Evaluate): Prove the Glivenko-Cantelli theorem and evaluate statistical influence functions for robust estimators.

0.0.0.3 Unit 3: Monte Carlo Methods (10 Lecture + 05 SDL + 10 Practicals = 25 Hours)

  • Apply Monte Carlo Integration (C3 — Apply): Estimate complex high-dimensional integrals using random sampling.
  • Master Variance Reduction Techniques (C6 — Create): Derive and apply Control Variates, Conditional Monte Carlo (Rao-Blackwellization), Stratified Sampling, and Multilevel Monte Carlo (MLMC).
  • Implement Importance Sampling & Particle Filters (C3 — Apply): Derive optimal proposal density \(g^*(x)\), Sequential Importance Sampling (SIS), and Resampling (SIR) for Hidden Markov Models.
  • Derive Score Function Gradients (C4 — Analyze): Utilize the log-derivative trick to compute expectation gradients.
  • Execute Resampling Methods (C3 — Apply): Perform Bootstrap, Jackknife, Cross-Validation, and Empirical Likelihood estimation.

0.0.0.4 Unit 4: Markov Chain Monte Carlo (05 Lecture + 05 SDL + 10 Practicals = 20 Hours)

  • Formulate MCMC Concepts (C3 — Apply): Construct ergodic Markov chains targeting unnormalized joint posterior distributions.
  • Derive Metropolis-Hastings & Gibbs Samplers (C5 — Create): Prove detailed balance, candidate acceptance ratios, and full conditional update dynamics.
  • Simulate Statistical Physics Models (C3 — Apply): Implement 2D Ising and Potts spin lattice models.
  • Execute Global Optimization & Exact Sampling (C5 — Create): Implement Simulated Annealing cooling schedules and Propp-Wilson Coupling From The Past (CFTP).
  • Apply Bayesian Inference & Kernel Density Estimation (C6 — Evaluate): Update posterior distributions and construct adaptive non-parametric KDE estimators.

0.0.0.5 Unit 5: Monte Carlo Optimization (03 Lecture + 03 SDL + 06 Practicals = 12 Hours)

  • Compare Optimization Frameworks (C4 — Analyze): Contrast deterministic and stochastic approximation algorithms.
  • Derive Robbins-Monro Stochastic Approximation (C6 — Create): Prove step-size convergence conditions for stochastic root-finding.
  • Execute Multilevel Splitting for Rare Events (C6 — Create): Implement fixed-factor and fixed-effort particle splitting algorithms.
  • Implement Adaptive Metropolis MCMC (C6 — Create): Construct Haario adaptive proposal schemes with diminishing adaptation limits.

Prerequisites & Setup

To run the interactive code blocks and exercises, source our shared helpers script in your R console:

source("helpers.R")

Statistical Research Methodology (DDS 6106) • Author: Dr. Shrikrishna Bhat K • Department of Applied Statistics and Data Science, PSPH, MAHE