Notes (Beta)

The notes on this website was written during my PhD in data science. Below are a list of topics.

• Supervised Learning: An Introduction
  • Least-squares
  • Nearest neighbors
  • Bias-variance tradeoff
  • Linear regression vs. kNN
  • Interval prediction
  • Review of Conditional Expectation
  • Exercises
• Discriminant Analysis
  • Overview
  • Details
  • Other notes
• Hidden Markov Models (HMM)
  • Baum Welch
• NN Learning
  • Useful Linear Algebra Identities
  • Gradient layout
  • Example on 1-layer NN
• Network science
  • Components of complex systems
  • Complexity Theory
  • Anatomy
  • Small World
  • Universality
  • Barabasi-Albert (BA) Model
• Generative Adversarial Networks (GANs)
  • Thought experiment: GANs for explainable AI?
• Explainable AI
• Summaries
  • ML & DL models
  • Sampling
  • Optimizers
• A/B Tests
  • 1. Set up the experiment
  • 2. Run the test and record the success rate for each group.
  • 3. Plot the distribution of the difference between the two samples.
  • 4. Calculate the statistical power.
  • 5. Evaluate how sample size affects A/B tests.
• Attention
• Common Questions
  • Vectors
  • Matrices
  • Linear regression
  • Dimensionality reduction
  • Statistics
• Cross Entropy
  • Negative log likelihood
• Regularization
  • Ridge
  • Lasso
  • Shrinkage effect
  • PCA
  • Partial Least Squares
  • Degrees of freedom
  • Lasso
• Theoretical Statistics
  • Probability Theory
  • Transformations and Expectations
  • Families of Distributions
  • Multiple Random Variables
  • Properties of Random Sample
  • Principles of Data Reduction
  • Point Estimation
  • Bayes
• Theoretical Statistics Questions
  • Is a pdf part of the exponential family?
  • Is a statistic complete?
  • Is a statistic sufficient?
  • Is a statistic unbiased?
  • Does a statistic converge in probability as \(n \xrightarrow{} \infty\)
  • Find unique best unbiased estimator of \(\theta\).
  • Find MLE
  • Find MOM
  • Find CRLB (variance bound)
  • Find statistic at CRLB
  • Best unbiased of \(\tau(\theta)\)
  • Find an LRT of size 0.05
  • Derive level \(\alpha\) UMP test of \(H_0\) and \(H_1\).
  • Find a pivot quantity and its distribution
  • Find a pivotal interval of \(\theta\) w/ confidence coeff \((1-\alpha)\)
  • Find smallest pivotal interval with CI \((1-\alpha)\)
• Natural Language Processing
• Graph neural networks at scale
  • Introduction
  • GNN Scaling issues
  • Solutions
  • Recent solutions in literature
• Contextual Bandits
  • Linear Bandits
  • Real-World Setting: Delayed Feedback
  • Conclusion
• Neural Network Gaussian Process (NNGP)
  • Summary
  • Definition
  • Proposition
  • Short proposition
  • Proof
  • Acknowledgements