Shubhashis
Roy Dipta
About
Publications
Resume
Projects
Notes
Awards
🧠 of Dipta
Search
Ctrl
+
K
🧠 of Dipta
Search
Ctrl
+
K
Home
Recent Notes
Literature Notes
Advanced NLP with Scipy
Deep Learning by Ian Goodfellow
DS & Algo Interview
How To 100M Learning Text Video
How to Read a Paper
How To Write a Paper
Papers
COIN
Compressed Chain of Thought - Efficient Reasoning Through Dense Representations
DeepSeek-R1
Deliberative Alignment - Reasoning Enables Safer Language Models
G-Eval - NLG Evaluation using GPT-4 with Better Human Alignment
Investigating Continual Pretraining in Large Language Models - Insights and Implications
Is a Question Decomposition Unit All We Need
Large Language Models are Zero-Shot Rankers for Recommender Systems
MM-LLMs
Molmo and PixMo
MultiVENT
OpenPI-C
Piecing It All Together - Verifying Multi-Hop Multimodal Claims
PubMedQA - A Dataset for Biomedical Research Question Answering
Scientific Fact-Checking - A Survey of Resources and Approaches
Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
tau-bench - A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Token Assorted - Mixing Latent and Text Tokens for Improved Language Model Reasoning
Vision Language Model-based Caption Evaluation Method Leveraging Visual Context Extraction
What is More Likely to Happen Next
Templates
Paper Template 1
Paper Template
Permanent Notes
Topic Template
Zotero Template
Topics
activation-function
algorithm
behavioral
deep-learning
evaluation
interview
llm
loss-in-ml
machine-learning
math
nlp
paper
probability
research
retrieval
sci-fact
statistics
vision
Zettelkasten
3 key question in data visualization
Accuracy
Activation Function
Active Learning
AdaBoost vs. Gradient Boosting vs. XGBoost
Adaboost
AdaDelta
AdaGrad
Adam
Adaptive Softmax
ADASYN
Additive Attention
Adjusted R-squared Value
Algorithm - Interview Scheduling
Alternative Hypothesis
Amazon Leadership Principles
Ancestral Sampling
Approximate Nearest Neighbor (ANN)
Area Under Curve (AUC)
Area Under Precision Recall Curve (AUPRC)
Attention
Auto Regressive Model
Autoencoder for Denoising Images
Autoencoder
Averaging in Ensemble Learning
Back Propagation
Backward Feature Elimination
Bag of Words
Bagging
Balanced Accuracy
Basics of Kubernetes
Batch Normalization
Bayes Theorem
Bayesian Optimization Hyperparameter Finding
Beam Search
Behavioral Interview
BERT Embeddings
BERT
Best Match 25 (BM25)
Bias & Variance
Bidirectional RNN or LSTM
Binary Cross Entropy
Binning or Bucketing
Binomial Distribution
bisect_left vs. bisect_right
bitsandbytes
BLEU Score
Boosting
Box Plot
Byte Level BPE
Byte Pair Encoding (BPE)
Cache Augmented Generation (CAG)
Causal Language Modeling
Causality vs. Correlation
Central Limit Theorem
Chain Rule
Challenges of NLP (2022)
Character Tokenizer
Choose the Right Statistical Test
Claim Verification Datasets
Claude Code
CNN
Co-occurrence based Word Embeddings
Co-Variance
Cold Start Problem
Collaborative Filtering
Collinearity
Combination
Conditional Probability
Conditional Random Field
Conditionally Independent Joint Distribution
Confusion Matrix
Connections - Log Likelihood, Cross Entropy, KL Divergence, Logistic Regression, and Neural Networks
Contextualized Word Embeddings
Continuous Bag of Words
Continuous Batching
Continuous Random Variable
Contrastive Learning
Contrastive Loss
Convex vs Nonconvex Function
Cosine Similarity
Count based Word Embeddings
Cross Entropy
Cross Validation
Cross-Attention
Crossed Feature
Curse of Dimensionality
DAPO
Data Augmentation
Data Imputation
Data Monitoring (DVC)
Data Visualization
DBScan Clustering
Debugging Deep Learning
Decision Boundary
Decision Tree (Classification)
Decision Tree (Regression)
Decision Tree
Decoder Only Transformer
Decoding Strategies
Density Sparse Data
Dependent Variable
Derivative
determinant
diagonal-matrix
Differentiation of Product
Differentiation
Digit Dp
Dimensionality Reduction
Discrete Random Variable
Discriminative vs. Generative Models
DistilBERT
do_sample (vllm vs hf)
doing-literature-review
Domain vs. Codomain vs. Range
Dropout
Dying ReLU
Dynamic Batching
Dynamic Programming (DP) in python
Early Stopping
Eigendecomposition
eigenvalue-eigenvector
Elastic Net Regression
ELMo Embeddings
Encoder Only Transformer
Encoder-Decoder Transformer
Ensemble Learning
Entropy and Information Gain
Entropy
Essential Visualizations
Estimated Mean
Estimated Standard Deviation
Estimated Variance
Euclidian Distance
Euclidian Norm
Exhaustive Search
Expected Value for Continuous Events
Expected Value for Discrete Events
Expected Value
Exploding Gradient
Exponential Distribution
Extrinsic Evaluation
F1 Score
F-Beta Score
F-Beta@K
Fake News Challenge
False Negative Error
False Positive Rate
FastText Embedding
Feature Engineering
Feature Extraction
Feature Hashing
Feature Preprocessing
Feature Selection
Finding Co-relation between two data or distribution
Fine Tuning Large Language Models
Floating Point Explained
Forward Feature Selection
Foundation Model
Frobenius Norm
Fully Independent Joint Distribution
Fully Joint Distribution
Gaussian Distribution
GBM
Generalized Discriminant Analysis (GDA)
Genetic Algorithm Hyperparameter Finding
Gini Impurity
Global Minima
GloVe Embedding
GPT-OSS
GPU Computation for LLM
Gradient Boost (Classification)
Gradient Boost (Regression)
Gradient Boosting
Gradient Checkpointing
Gradient Clipping
Gradient Descent
Gradient
Graph Convolutional Network (GCN)
Greedy Decoding
Grid Search Cross Validation
Grid Search Hyperparameter Finding
Group Normalization
Group-Query Attention
GRPO
GRU
GSPO
Gumbel Softmax
Handling Imbalanced Dataset
Handling Missing Data
Handling Outliers
Hard Margin in SVM
Heapq (nlargest or nsmalles)
Hidden Markov Model
Hierarchical Clustering
Hierarchical Softmax
Hinge Loss
Histogram
Homonym or Polysemy
How to Choose Kernel in SVM
How to cite paper?
How to combine in Ensemble Learning
How to create a Scientific Poster
How to do research
How to prepare for Behavioral Interview
How to Write Academic Paper (from CS Perspective)
Huber Loss
Hyperparameters
Hypothesis Testing
Hypothetical Document Embedding (HyDE)
Identity Matrix
Implement Linear Regression using Numpy
Independent Component Analysis (ICA)
Independent Variable
InfoNCE Loss
Instruction Fine Tuning
Instructional Websites
Integration by Parts or Integration of Product
Internal Covariate Shift
Interquartile Range (IQR)
Interview Resources
Intrinsic Evaluation
Jaccard Distance
Jaccard Similarity
Jacobian Matrix
Joint Distribuition
Joint Probability
jupyter-notebook-on-server
K Fold Cross Validation
K-means Clustering
K-means vs. Hierarchical
K-nearest Neighbor (KNN)
Kernel in SVM
Kernel Regression
Kernel Trick
KL Divergence
KTO
KV Cache
L1 or Lasso Regression
L1 vs. L2 Regression
L2 or Ridge Regression
Label Encoding
Label Smoothing
Layer Normalization
Leaky ReLU
Learning Rate Scheduler
Learning Rate
Learning to Rank
Leave one out Cross Validation
Lemmatization
LightGBM
Likelihood
Line Equation
Linear Discriminant Analysis (LDA)
Linear Regression with Normal Equation
Linear Regression
Listwise Ranking
LLM GPU Calculate
Local Attention
Local Minima
Log (Odds Ratio)
Log (Odds)
Log Normalization
Log Scale
Log-cosh Loss
logarithm
Logistic Regression vs. Neural Network
Logistic Regression
lp-norm
LSTM
Machine Learning Algorithm Selection
Machine Learning vs. Deep Learning
Majority vote in Ensemble Learning
Mamba Architecture
Manhattan Distance
Margin in SVM
Marginal Probability
Masked Language Modeling
Masked Self-Attention
Math Dataset
matplotlib functions
matplotlib legend
Matrices
Matrix Factorization
Max Norm
Maximal Margin Classifier
Maximum Likelihood
Mean Absolute Error (MAE)
Mean Absolute Percentage Error (MAPE)
Mean Reciprocal Rank (MRR)
Mean Squared Error (MSE)
Mean Squared Logarithmic Error (MSLE)
Mean
Median
Merge K-sorted List
Merge Overlapping Intervals
Meteor Score
Min Max Normalization
Mini Batch SGD
Mixed Precision
Mixture of Experts
ML Case Study or ML Design
ML System Design
Mode
Model Based vs. Instance Based Learning
Multi Class Cross Entropy
Multi Label Cross Entropy
Multi Layer Perceptron
Multi-Head Attention
Multi-Head Latent Attention
Multi-Query Attention
Multicollinearity
Multiset
Multivariable Linear Regression
Multivariate Linear Regression
Multivariate Normal Distribution
Mutual Information
N-gram Method
Naive Bayes
Named Entity Recognition (NER)
NDCG
Negative Log Likelihood
Negative Sampling
Nesterov Accelerated Gradient (NAG)
Neural Network Normalization
Neural Network
Next Sentence Prediction
Norm of a Vector
Normal Distribution
Normalization
Null Hypothesis
Odds Ratio
Odds
One Class Classification
One Class Gaussian
One Hot Vector
One vs One Multi Class Classification
One vs Rest or One vs All Multi Class Classification
Optimizers
Optimizing Transformer
Orthogonal Matrix
Orthonormal Vector
Overcomplete Autoencoder
Overfitting
Oversampling
p Value
Padding in CNN
Paged KV Cache
Pairwise Ranking
Papers Must Read
Parallelism in LLM
Parameter vs. Hyperparameter
PCA vs. Autoencoder
Pearson Correlation
Perceptron
Permutation
Perplexity
Personal Claude Code Recommendations
Plots Compared
Pointwise Ranking
Polynomial Kernel
Polynomial Regression
Pooling
Population
Positional Encoding in Transformer
Posterior Probability
Pre-Fill in LLM
Pre-Training LLM
Precision Recall Curve (PRC)
Precision
Precision@K
Prepare for Talk
Presentation Making Tips
Principal Component Analysis (PCA)
Prior Probability
Probability Density Function
Probability Distribution
Probability Mass Function
Probability vs. Likelihood
Problem Solving Algorithm Selection
Prompt Engineering
Proximal Policy Optimization (PPO)
Pruning in Decision Tree
PyTorch Loss Functions
PyTorch Refresher
QLORA
Quantization Technique
Questions to ask in a Interview?
Quintile or Percentile
Quotient Rule or Differentiation of Division
R-squared Value
Radial Basis Kernel
Random Forest
Random Search Cross Validation
Random Variable
Recall
Recall@K
Recommender System (RecSys)
Regularization
Reinforcement Learning from Human Feedback (RLHF)
Reinforcement Learning
Relational GCN
ReLU
Reno Talk @UMBC on Scale-2024
Research Poster
Research Skills Unsorted List
Research Slides or Research Talk
Retrieval Metrics
Retrieval Ranking Funnel
RMSProp
RNN
Robust Scaling Normalization
ROC Curve
Root Mean Squared Error (RMSE)
Root Mean Squared Logarithmic Error (RMSLE)
Rotary Position Embedding (RoPE)
ROUGE-L Score
ROUGE-LSUM Score
ROUGE-N Score
RTE (Recognizing Textual Entailment)
Saddle Points
Scalar
Second Order Derivative or Hessian Matrix
Self-Attention
Self-Supervised Learning
Semi-supervised Learning
Sensitivity
SentencePiece Tokenization
Sequence-to-Sequence Model
Sigmoid Function
Sigmoid Kernel
Simple Linear Regression
Singular Value Decomposition (SVD)
Skip Gram Model
Sliding KV Cache
Sliding Window Attention
SMOTE
Soft Margin in SVM
Softmax
Softplus
Softsign
Some Common Behavioral Questions
Sources of Uncertainty
spacy-doc-object
spacy-doc-span-token
spacy-explanation-of-labels
spacy-matcher
spacy-named-entities
spacy-operator-quantifier
spacy-pattern
spacy-pipeline
spacy-pos
spacy-semantic-similarity
spacy-syntactic-dependency
Sparse Mixture of Experts
Specificity
Splitting tree in Decision Tree
Stacking or Meta Model in Ensemble Learning
Standard deviation
Standardization
State Space Model
Statistical Power
Statistical Significance
Stemming
Stepwise Selection
Stochastic Gradient Descent (SGD)
Stochastic Gradient Descent with Momentum
Stop Words
STORM Method
Stratified K Fold Cross Validation
Stride in CNN
Stump
Sub-sampling in Word2Vec
Sub-word Tokenizer
Supervised Learning
Support Vector Machine (SVM)
Support Vector
Surprise
SVC
Swallow vs. Deep Learning
t-SNE
Tanh
Temperature in Decoding
Text Preprocessing
TF-IDF
Three Way Partioning
Time Complexity of ML Algos
Time Complexity of ML Models
Tokenizer
Top-K in Retrieval System
Toward RL Learning
Trace Operator
Training a Deep Neural Network
Transformer vs LSTM
Transformer
Triplet Loss
True Negative Rate
True Positive Rate
Two Pointer
Two-Tower Model
Type 1 Error vs. Type 2 Error
Undercomplete Autoencoder
Undersampling
Uniform Distribution
Unigram Tokenization
Unit Vector
Unsupervised Learning
Untitled
Vanishing Gradient in Transformers
Vanishing Gradient
Variance
Variational Autoencoder
Vector Database
Vector
Vision Transformer
Weakly Supervised Learning
Weight Initialization
When less data is better than more?
Why do we scale attention weights?
Why do we use Projection in QKV?
Why Trigonometric Function for Positional Encoding?
Word2Vec Embedding
Word Embeddings
Word Error Rate
Word Tokenizer
WordPiece Tokenization
XGBoost
Yet another Rope Extension (YaRN)
Z-Score Normalization
Zotero Guides
Enter your search text in the box above
Select a result to preview
Softplus
#deep-learning
#activation-function
#interview
Smoother version of
ReLU
Negative values still get 0
But for positive ones, its smoother and unbounded
Improve training stability
But computation is expensive due to complex gradient
Softplus Formula
f
(
x
)
=
l
o
g
(
1
+
exp
(
x
)
)
Related Notes
Early Stopping
Standardization
Graph Convolutional Network (GCN)
Skip Gram Model
Bidirectional RNN or LSTM
Ask my research
Ask my research
AI answers about my work & background
New chat
Hi! Ask me anything about
Shubhashis Roy Dipta
- his research, papers, experience, and background.
What's his research about?
Recent first-author papers?
What did he do at Amazon & Scale AI?
Why should someone hire him?
AI-generated, may be imperfect. Email
sroydip1@umbc.edu
for anything important.
