A curated collection of papers, articles, and resources I found interesting during Summer 2023. This list spans across machine learning, AI safety, adversarial robustness, large language models, and various other topics in AI research. These resources helped shape my understanding of current challenges and opportunities in the field.
- SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
- Dataset Condensation with Distribution Matching
- Convolution for Computer Science People
- Score-Based Diffusion Models | Fan Pu Zeng
- NVIDIA LLM Developer AI Day
- PoisonGPT: How to poison LLM supply chain on Hugging Face
- Hamming, "You and Your Research" (June 6, 1995)
- sigstore
- GitHub - andyzoujm/representation-engineering: Representation Engineering: A Top-Down Approach to AI Transparency
- Representation Engineering: A Top-Down Approach to AI Transparency
- GitHub - guardrails-ai/guardrails: Adding guardrails to large language models.
- Guardrails AI | Your Enterprise AI needs Guardrails
- MetNet-3: A state-of-the-art neural weather model available in Google products
- chiphuyen's list / Cool LLM repos
- Idempotent Generative Network
- PromptIDE
- Adversarial Attacks on LLMs
- Evaluation & Hallucination Detection for Abstractive Summaries
- What is a Vector Database & How Does it Work? Use Cases + Examples | Pinecone
- Introducing Pika 1.0, An Idea-to-Video Platform
- My North Star for the Future of AI
- Gemini - Google DeepMind
- Consistency Models
- The Gemini Lie
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Perspectives on the State and Future of Deep Learning - 2023
- WhiteRabbitNeo/WhiteRabbitNeo-13B-v1 · Hugging Face
- Archives - colah's blog
- Developing Llama 2 | Angela Fan
- Double descent - Wikipedia
- Highly accurate protein structure prediction with AlphaFold - Nature
- Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models
- Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors
- skfolio
- Optimize PyTorch Performance for Speed and Memory Efficiency (2022) | by Jack Chih-Hsu Lin | in Towards Data Science - Freedium
- AlphaGeometry: An Olympiad-level AI system for geometry
- The Faiss library
- Generative Agents: Interactive Simulacra of Human Behavior
- UniVTG: Towards Unified Video-Language Temporal Grounding
- HOW HARD IS TROJAN DETECTION IN DNNS? FOOLING DETECTORS WITH EVASIVE TROJANS
- Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms
- 4 Autonomous AI Agents you need to know
- Image Restoration with Mean-Reverting Stochastic Differential Equations
- CS25 I Stanford Seminar 2022 - Transformer Circuits, Induction Heads, In-Context Learning
- LLM Attacks
- AIM and continuous value data could transform computing
- Adversarial Examples Are Not Bugs, They Are Features
- On Adaptive Attacks to Adversarial Example Defenses
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- Llama2
- Attention is Turing Complete
- The Best Defense is a Good Offense: Adversarial Augmentation against Adversarial Attacks
- Perspectives on diffusion
- Deep Dive into Kernel Fusion: Accelerating Inference in Llama V2 - Lefebvre Sarrut's AI blog
- Keras: Deep Learning for humans
- Whose responsibility is responsible AI?
- LLM trojan
- Tight Auditing of Differentially Private Machine Learning
- Adversarial training and robustness for multiple perturbations
- No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"
- "Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice
- Poisoning Web-Scale Training Datasets is Practical
- Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators
- Label-Only Membership Inference Attacks
- New ways of breaking app-integrated LLMs
- Grammatical Error Correction: Tag, Not Rewrite
- How we built it: Stripe Radar
- Inside GitHub: Working with the LLMs behind GitHub Copilot | The GitHub Blog
- Reconstructing indoor spaces with NeRF
- Xerox scanners/photocopiers randomly alter numbers in scanned documents
- Navigating the Challenges of LLMs: Guardrails AI to the Rescue
- Parameter-Free Optimizers for Pytorch
- Sponge Examples: Energy-Latency Attacks on Neural Networks
- Washing The Unwashable : On The (Im)possibility of Fairwashing Detection
- The Security Hole at the Heart of ChatGPT and Bing
- Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA
- PyPI Repository Under Attack
- Global and surrogate methods, interpretable models
- Local post hoc methods
- Writing Python like it's Rust
- SAP/ml-model-watermarking
- Where is the Information in a Deep Neural Network?
- Confident Learning: Estimating Uncertainty in Dataset Labels
- Compromised PyTorch Dependency Chain
- Machine Language Modelling from System Logging
- A tutorial on Differential Evolution with Python
- Faster Deep Learning Training with PyTorch – a 2021 Guide
- Model Calibration
- Investigating the Nature of 3D Generalization in Deep Neural Networks
- ImageBind: One Embedding Space To Bind Them All
- Unlimiformer: Long-Range Transformers with Unlimited Length Input
- Product Launch 2023 Keynote
- Interpretability of Transformers with up to two layers of attention
- Using Softmax Linear Units(SoLU) to investigate interpretability of transformers
- Beyond automatic differentiation
- Cultivating Your Research Taste
- Choose Your Weapon: Survival Strategies for Depressed AI Academics
- Approximating Wasserstein distances with PyTorch
- Stochastic Weight Averaging — a New Way to Get State of the Art Results in Deep Learning
- 30B model now needs only 5.8GB of RAM? How?
- Ilya Sutskever (OpenAI Chief Scientist) - Building AGI, Alignment, Spies, Microsoft, & Enlightenment
- Watermarking for Out-of-distribution Detection
- Continual Few-Shot Learning Using HyperTransformers
- NeurIPS 2022 Workshop MLSW Submissions
- Creating Confidence Intervals for Machine Learning Classifiers
- 26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone
- tinygrad: A simple and powerful neural network framework
- GPT in 60 Lines of NumPy | Jay Mody
- System 2 Is What We Need
- Quick tour - BlindLlama
- Validating LLM Outputs
- The Rise and Potential of Large Language Model Based Agents: A Survey
- GitHub - laiyer-ai/llm-guard: The Security Toolkit for LLM Interactions
- Laiyer: Unleash LLM's potential with confidence
- Introduction to AI Accountability & Transparency Series
- FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
- A tale of two problem solvers (Average cube shadows)
- From Newton's method to Newton's fractal (which Newton knew nothing about)
- Full Event | #MicrosoftEvent September 21, 2023
- The Adventure of the Errant Hardware
- A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
- AutoPrompt
- Work & Projects Summary | CAIS
- Scaling up learning across many different robot types
- Rewind Pendant
- NVIDIA Technical Blog | News and tutorials for developers, data scientists, and IT admins
- Google Colaboratory
- Neel Nanda
- Generating Synthetic Dataset for RAG – Nextra
- The AI research job market shit show (and my experience)
- keerthanapg
- AI's Underbelly: The Zero-Day Goldmine by: Dan McInerney
- huntr - The world's first bug bounty platform for AI/ML
- GitHub - jxmorris12/vec2text: utilities for decoding deep representations (like sentence embeddings) back to text
- Compiling NumPy code into C++ or CUDA via torch.compile
This collection represents a diverse range of topics from cutting-edge AI research to practical implementation guides. Many of these resources helped shape my understanding of current challenges and opportunities in machine learning, particularly around model robustness, AI safety, and the responsible deployment of large language models.
The list includes everything from foundational papers on adversarial robustness and model interpretability to practical guides on optimizing deep learning workflows. I found the resources on AI safety and responsible AI deployment particularly valuable as the field grapples with the rapid advancement of large language models and their societal implications.
Some standout resources include the work on representation engineering for AI transparency, various approaches to defending against adversarial attacks on LLMs, and insights into the practical challenges of deploying robust ML systems in production environments.