Papers Shelf
A collection of some papers on various topics. Keeping an organized list helps in building mental models and quickly remembering the crux of the work.
Stochastic PDE’s
- How to solve the stochastic partial differential equation that gives a Matérn random field using the finite element method - Haakon Bakka (2018) - A method for solving SPDEs giving Matérn random fields using the finite element method.
Distributional Regression
- Neural Additive Models: Interpretable Machine Learning with Neural Nets - Agarwal, R., Frosst, N., Zhang, X., Caruana, R., Hinton, G. (2020) - Introduces interpretable neural nets for regression tasks.
- NAMLSS: Neural Additive Models for Location Scale and Shape - Thielmann, A., Kruse, R., Kneib, T., Safken, B. (2023) - Focuses on distributional regression using machine learning methods.
- Generalized Additive Models for Location, Scale, and Shape for High Dimensional Data - Andreas Mayr et al. (2012) - Boosting techniques for high-dimensional data.
- GAMLSS: Generalized Additive Models for Location Scale and Shape - Stasinopoulos, D. Mikis, Rigby, R. A. (2007) - Foundational work in distributional regression.
- Rage Against the Mean – A Review of Distributional Regression Approaches - Kneib, T., Silbersdorff, A., Säfken, B. (2023) - A comprehensive review of distributional regression methods.
- Semi-Structured Distributional Regression Paper - Combines structured regression models with deep neural networks, addressing identifiability issues with an orthogonalization cell.
Interpretable Deep Learning
- NODE: Neural Oblivious Decision Ensembles - Popov S., Stanislav S., Babenko A. (2019) - Generalizes ensembles of oblivious decision trees, with gradient-based optimization.
- NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learning - Chang, C., Caruana, R., Goldenberg, A. (2021) - Combines NODE and GAMs.
- NBM: Neural Basis Models - A novel model combining neural networks with basis expansions for interpretability.
- SPAM: Scalable Polynomial Additive Models - Uses tensor rank decompositions of polynomials for interpretable yet powerful modeling.
- SNAM (Structural Neural Additive Models) - Uses splines instead of MLPs in NAMs to fit neural models and optimize knot locations, enhancing interpretability.
- NAIM (Neural Additive Image Models) - Combines NAMs with diffusion autoencoders to identify latent image semantics and study complex image effects.
- NATT (Neural Additive Tabular Transformer Networks) - Merges additive neural networks with Transformer models to model categorical features using Transformer encoders.
- Sparse NAM - Enhances NAMs with group sparsity regularization for feature selection and generalization improvements.
- Sparse Interaction Additive Networks - Identifies necessary feature combinations, optimizing the balance between network complexity and model generalizability.
Regression Models
- A review and recommendations on variable selection methods in regression models for binary data - Souvik Bag, Kapil Gupta, Soudeep Deb - Reviews variable selection methods in binary data regression.
Variational Inference
- Variational Inference I - Course Notes (2011) - Introduction to variational inference techniques.
- A Stochastic Approximation Method - Robbins, H., Monro, S. (1951) - A foundational method for stochastic approximation.
- Variational Inference: A Review for Statisticians - Blei, D. M., Kucukelbir, A., McAuliffe, J. D. (2017) - A comprehensive review of variational inference methods.
- Logistic VB - Michael Komodromos, Marina Evangelou, Sarah Filippi (2024): An approximation for integral (log(1+e^x)N(mu, sigma^2)).
- Variational Inference of Sparse Network from Count Data - Julien Chiquet (2019) - Presents an alternate optimization technique for network sparsity.
Graphical Models, Bayesian Networks and Causality
- Causal Inference in Statistics: An overview - Pearl (2009)
- Elements of Causal Inference - Jonas Peters
- glasso - Graphical Lasso