Topic modelling

Topic Modelling is a clustering technique used to classify documents (text) according to the topic they deal with. It is an unsupervised machine learning technique. Let's go through topic modelling step by step using a text data set. We will be using articles from NPR (National Public Radio), obtained from their website www.npr.org Intuition How would...

Stationarity in time series data – What, Why and How

Before stationarity, let's understand white noise. What is white noise? A white noise series is a sequence of random numbers and cannot be predicted. More formally, a series is white noise if the variables are independent and identically distributed with mean of zero and same variance. Each value has zero correlation with other values in...

Hypothesis Testing in Linear Regression

If you need some introduction to Linear Regression, Go here ! https://vipanchiks.wordpress.com/2022/05/28/linear-regression-a-detailed-introduction/ You must be familiar with the output of a linear regression model. We get a p - value for each variable. What is this p-value? Hypothesis Testing : For a simple linear regression, as you know, the equation will look like this: If...

Linear Regression – A detailed introduction

What is Regression? We use this term very often in Machine Learning and Statistics. What is the meaning of this term? It is a method used in statistics to determine the relation between on variable (dependent) and the other variable (independent). Literal meaning of regression is stepping back towards the average. So , where from...

Understanding SST – ANOVA

When population means are same / similar : When the population means are same or nearer, SSW will be high, in other words, most of SST will go into SSW . Checking it in python : See that, SSW is more compared to SSB When population means are different: When population means are different, then...

Multicollinearity – A detailed Understanding

Multicollinearity: It is the existence of correlation among the predictor variables. Multicollinearity is the occurrence of high intercorrelations among two or more independent variables in a multiple regression model.Investopedia Why is it a problem? Multicollinearity among independent variables will result in less reliable statistical inferences. Multicollinearity increases the variability in coefficients, making the estimates sensitive to...

Math in ANOVA – Explained

If you don't know what ANOVA is, no worries. Got you covered! https://vipanchiks.wordpress.com/2022/05/17/anova-analysis-of-variance-introduction/ Mathematically understanding ANOVA: The purpose of ANOVA is to compare the population means, remember? How do we do that? Lets dive in! Method: Lets say we have r populations. So, r samples. Calculate the mean of each sample : xi Now, combine...

ANOVA – Analysis of Variance: Introduction

To carryout comparison of means of several populations we use Analysis of Variance. ANOVA is a statistical method for determining the existence of differences among several population means Aczel - Sounderpandian Why is it called Analysis of Variance? Though we are comparing different population means for difference, the technique requires the analysis of different forms...

Hypothesis Testing – Introduction

A hypothesis is an statement which can be tested for truth. If someone claims that average stress levels among students of a college are greater than equal to 6, we can't go by that statement can we? We have to check if the statement is true or false. How do we do that? Intuition: A...

Confidence Interval

If you want to make an inference on the population parameter from a sample statistic., then you use this concept. Let's say we want to find the average cost of PGs all over HSR layout. Instead of going to every PG (population) and finding the mean price, we pick a sample randomly and find the...