About
Publications
Resume
Projects
Notes
Awards

🧠 of Dipta

Search Ctrl + K

🧠 of Dipta

Search Ctrl + K

Home

Recent Notes

Literature Notes

Advanced NLP with Scipy

Deep Learning by Ian Goodfellow

DS & Algo Interview

How To 100M Learning Text Video

How to Read a Paper

How To Write a Paper

Papers

COIN

Compressed Chain of Thought - Efficient Reasoning Through Dense Representations

DeepSeek-R1

Deliberative Alignment - Reasoning Enables Safer Language Models

G-Eval - NLG Evaluation using GPT-4 with Better Human Alignment

Investigating Continual Pretraining in Large Language Models - Insights and Implications

Is a Question Decomposition Unit All We Need

Large Language Models are Zero-Shot Rankers for Recommender Systems

Piecing It All Together - Verifying Multi-Hop Multimodal Claims

PubMedQA - A Dataset for Biomedical Research Question Answering

Scientific Fact-Checking - A Survey of Resources and Approaches

Semantic Product Search for Matching Structured Product Catalogs in E-Commerce

tau-bench - A Benchmark for Tool-Agent-User Interaction in Real-World Domains

Token Assorted - Mixing Latent and Text Tokens for Improved Language Model Reasoning

Vision Language Model-based Caption Evaluation Method Leveraging Visual Context Extraction

What is More Likely to Happen Next

Templates

Topics

llm

math

nlp

paper

vision

Zettelkasten

3 key question in data visualization

Accuracy

Activation Function

Active Learning

AdaBoost vs. Gradient Boosting vs. XGBoost

Adam

ADASYN

Adjusted R-squared Value

Algorithm - Interview Scheduling

Alternative Hypothesis

Amazon Leadership Principles

Ancestral Sampling

Approximate Nearest Neighbor (ANN)

Area Under Curve (AUC)

Area Under Precision Recall Curve (AUPRC)

Attention

Auto Regressive Model

Autoencoder for Denoising Images

Autoencoder

Averaging in Ensemble Learning

Back Propagation

Backward Feature Elimination

Bayesian Optimization Hyperparameter Finding

BERT

Bidirectional RNN or LSTM

Binary Cross Entropy

Binning or Bucketing

Binomial Distribution

bisect_left vs. bisect_right

Byte Pair Encoding (BPE)

Cache Augmented Generation (CAG)

Causal Language Modeling

Causality vs. Correlation

Central Limit Theorem

Chain Rule

Challenges of NLP (2022)

Character Tokenizer

Choose the Right Statistical Test

Claim Verification Datasets

Claude Code

CNN

Co-occurrence based Word Embeddings

Co-Variance

Cold Start Problem

Collaborative Filtering

Collinearity

Combination

Conditional Probability

Conditional Random Field

Conditionally Independent Joint Distribution

Confusion Matrix

Connections - Log Likelihood, Cross Entropy, KL Divergence, Logistic Regression, and Neural Networks

Contextualized Word Embeddings

Continuous Bag of Words

Continuous Batching

Continuous Random Variable

Contrastive Learning

Contrastive Loss

Convex vs Nonconvex Function

Cosine Similarity

Count based Word Embeddings

Curse of Dimensionality

DAPO

Data Augmentation

Data Imputation

Data Monitoring (DVC)

Data Visualization

DBScan Clustering

Debugging Deep Learning

Decision Boundary

Decision Tree (Classification)

Decision Tree (Regression)

Decision Tree

Decoder Only Transformer

Differentiation of Product

Differentiation

Digit Dp

Dimensionality Reduction

Discrete Random Variable

Discriminative vs. Generative Models

DistilBERT

do_sample (vllm vs hf)

doing-literature-review

Domain vs. Codomain vs. Range

Dropout

Dying ReLU

Dynamic Batching

Dynamic Programming (DP) in python

Early Stopping

Eigendecomposition

eigenvalue-eigenvector

Elastic Net Regression

ELMo Embeddings

Encoder Only Transformer

Encoder-Decoder Transformer

Ensemble Learning

Entropy and Information Gain

Entropy

Essential Visualizations

Estimated Mean

Estimated Standard Deviation

Expected Value for Continuous Events

Expected Value for Discrete Events

Expected Value

Exploding Gradient

Exponential Distribution

Feature Preprocessing

Feature Selection

Finding Co-relation between two data or distribution

Fine Tuning Large Language Models

Floating Point Explained

Forward Feature Selection

Foundation Model

Frobenius Norm

Fully Independent Joint Distribution

Fully Joint Distribution

Gaussian Distribution

GBM

Generalized Discriminant Analysis (GDA)

Genetic Algorithm Hyperparameter Finding

GPU Computation for LLM

Gradient Boost (Classification)

Gradient Boost (Regression)

Gradient Boosting

Gradient Checkpointing

Gradient Clipping

Gradient Descent

Gradient

Graph Convolutional Network (GCN)

Greedy Decoding

Grid Search Cross Validation

Grid Search Hyperparameter Finding

Group Normalization

Group-Query Attention

GRPO

GRU

GSPO

Gumbel Softmax

Handling Imbalanced Dataset

Handling Missing Data

Handling Outliers

Hard Margin in SVM

Heapq (nlargest or nsmalles)

Hidden Markov Model

Hierarchical Clustering

How to Choose Kernel in SVM

How to cite paper?

How to combine in Ensemble Learning

How to create a Scientific Poster

How to do research

How to prepare for Behavioral Interview

How to Write Academic Paper (from CS Perspective)

Huber Loss

Hyperparameters

Hypothesis Testing

Hypothetical Document Embedding (HyDE)

Identity Matrix

Implement Linear Regression using Numpy

Independent Component Analysis (ICA)

Independent Variable

InfoNCE Loss

Instruction Fine Tuning

Instructional Websites

Integration by Parts or Integration of Product

Internal Covariate Shift

Interquartile Range (IQR)

jupyter-notebook-on-server

K Fold Cross Validation

K-means Clustering

K-means vs. Hierarchical

K-nearest Neighbor (KNN)

KTO

L1 or Lasso Regression

L1 vs. L2 Regression

L2 or Ridge Regression

Learning Rate Scheduler

Learning Rate

Learning to Rank

Leave one out Cross Validation

Linear Discriminant Analysis (LDA)

Linear Regression with Normal Equation

Logistic Regression vs. Neural Network

Logistic Regression

lp-norm

LSTM

Machine Learning Algorithm Selection

Machine Learning vs. Deep Learning

Majority vote in Ensemble Learning

Masked Language Modeling

Masked Self-Attention

Maximal Margin Classifier

Maximum Likelihood

Mean Absolute Error (MAE)

Mean Absolute Percentage Error (MAPE)

Mean Reciprocal Rank (MRR)

Mean Squared Error (MSE)

Mean Squared Logarithmic Error (MSLE)

Mean

Median

Merge K-sorted List

Merge Overlapping Intervals

Meteor Score

Min Max Normalization

Mini Batch SGD

Mixed Precision

Mixture of Experts

ML Case Study or ML Design

ML System Design

Mode

Model Based vs. Instance Based Learning

Multi Class Cross Entropy

Multi Label Cross Entropy

Multi Layer Perceptron

Multi-Head Attention

Multi-Head Latent Attention

Multi-Query Attention

Multicollinearity

Multiset

Multivariable Linear Regression

Multivariate Linear Regression

Multivariate Normal Distribution

Mutual Information

N-gram Method

Naive Bayes

Named Entity Recognition (NER)

NDCG

Negative Log Likelihood

Negative Sampling

Nesterov Accelerated Gradient (NAG)

Neural Network Normalization

Neural Network

Next Sentence Prediction

Odds

One Class Classification

One Class Gaussian

One Hot Vector

One vs One Multi Class Classification

One vs Rest or One vs All Multi Class Classification

Optimizers

Optimizing Transformer

Orthogonal Matrix

Orthonormal Vector

Overcomplete Autoencoder

Parameter vs. Hyperparameter

Personal Claude Code Recommendations

Plots Compared

Pointwise Ranking

Polynomial Kernel

Polynomial Regression

Pooling

Population

Positional Encoding in Transformer

Posterior Probability

Pre-Fill in LLM

Pre-Training LLM

Precision Recall Curve (PRC)

Precision

Precision@K

Prepare for Talk

Presentation Making Tips

Principal Component Analysis (PCA)

Prior Probability

Probability Density Function

Probability Distribution

Probability Mass Function

Probability vs. Likelihood

Problem Solving Algorithm Selection

Prompt Engineering

Proximal Policy Optimization (PPO)

Pruning in Decision Tree

PyTorch Loss Functions

PyTorch Refresher

QLORA

Quantization Technique

Questions to ask in a Interview?

Quintile or Percentile

Quotient Rule or Differentiation of Division

R-squared Value

Radial Basis Kernel

Random Forest

Random Search Cross Validation

Random Variable

Recall

Recall@K

Recommender System (RecSys)

Regularization

Reinforcement Learning from Human Feedback (RLHF)

Reinforcement Learning

Relational GCN

ReLU

Reno Talk @UMBC on Scale-2024

Research Poster

Research Skills Unsorted List

Research Slides or Research Talk

Retrieval Metrics

Retrieval Ranking Funnel

RMSProp

RNN

Robust Scaling Normalization

ROC Curve

Root Mean Squared Error (RMSE)

Root Mean Squared Logarithmic Error (RMSLE)

Rotary Position Embedding (RoPE)

ROUGE-L Score

ROUGE-LSUM Score

ROUGE-N Score

RTE (Recognizing Textual Entailment)

Saddle Points

Scalar

Second Order Derivative or Hessian Matrix

Self-Attention

Self-Supervised Learning

Semi-supervised Learning

Sensitivity

SentencePiece Tokenization

Sequence-to-Sequence Model

Sigmoid Function

Sigmoid Kernel

Simple Linear Regression

Singular Value Decomposition (SVD)

Skip Gram Model

Sliding KV Cache

Sliding Window Attention

SMOTE

Some Common Behavioral Questions

Sources of Uncertainty

spacy-doc-object

spacy-doc-span-token

spacy-explanation-of-labels

spacy-matcher

spacy-named-entities

spacy-operator-quantifier

spacy-pattern

spacy-pipeline

spacy-pos

spacy-semantic-similarity

spacy-syntactic-dependency

Sparse Mixture of Experts

Specificity

Splitting tree in Decision Tree

Stacking or Meta Model in Ensemble Learning

Statistical Significance

Stemming

Stepwise Selection

Stochastic Gradient Descent (SGD)

Stochastic Gradient Descent with Momentum

Stop Words

STORM Method

Stratified K Fold Cross Validation

Stride in CNN

Stump

Sub-sampling in Word2Vec

Sub-word Tokenizer

Supervised Learning

Support Vector Machine (SVM)

Support Vector

Surprise

SVC

Swallow vs. Deep Learning

t-SNE

Tanh

Temperature in Decoding

Text Preprocessing

TF-IDF

Three Way Partioning

Time Complexity of ML Algos

Time Complexity of ML Models

Tokenizer

Top-K in Retrieval System

Toward RL Learning

Trace Operator

Training a Deep Neural Network

Type 1 Error vs. Type 2 Error

Undercomplete Autoencoder

Unsupervised Learning

Untitled

Vanishing Gradient in Transformers

Vanishing Gradient

Variance

Variational Autoencoder

Vector Database

Vector

Vision Transformer

Weakly Supervised Learning

Weight Initialization

When less data is better than more?

Why do we scale attention weights?

Why do we use Projection in QKV?

Why Trigonometric Function for Positional Encoding?

WordPiece Tokenization

XGBoost

Yet another Rope Extension (YaRN)

Z-Score Normalization

Zotero Guides

Enter your search text in the box above

Select a result to preview

Enter to select

⇅ to navigate

ESC to close

Sparse Mixture of Experts

#deep-learning #llm #interview #todo

References

Related Notes

Quantization Technique
How to create a Scientific Poster
Mamba Architecture
Vanishing Gradient in Transformers
Stepwise Selection

Connected Pages

Sparse Mixture of Experts

References

Pages mentioning this page

deep-learning

interview

llm

Ask my research AI answers about my work & background

Hi! Ask me anything about Shubhashis Roy Dipta - his research, papers, experience, and background.

AI-generated, may be imperfect. Email sroydip1@umbc.edu for anything important.