Databases

Broadly, a database is a large, integrated collection of data stored and accessed electronically.

A database management system (DBMS) is a software system designed to store, manage, and facilitate access to databases.

Data is at the heart of most significant programs, but few understand how database systems actually work.

I use:

• PostgreSQL
• SQlite
• MongoDB
• ClickHouse
• RocksDB
• LevelDB
• RethinkDB
• TimescaleDB
• InfluxDB
• Prometheus
• etcd
• Firestore
• Citus
• Redis
• Elasticsearch
• Apache Kafka
• Supabase
• Prisma

Links

• Things you should know about databases
• Things I Wished More Developers Knew About Databases (2020)
• Databases in 2022: A Year in Review
• Building a database in the 2020s
• The B-Tree, LSM-Tree, and the Bw-Tree in Between
• A Database Without Dynamic Memory Allocation
• Retrospection and Learnings from Dgraph Labs
• How Discord Stores Billions of Messages
• Reddit's database has two tables
• Overview of Consistency Levels in Database Systems
• Soft Deletion Probably Isn't Worth It
• Offline data access: a dream come true?
• Code in database vs. code in application
• Ideas on better database design
• Super-Structured Data - Rethinking the schema.
• A Gentle Introduction to Vector Databases
• 42 things I learned from building a production database
• A terrible schema from a clueless programmer
• Spending $5k to learn how database indexes work
• How we built a forever-free serverless SQL database
• Offline First Database Comparison - An implementation of the exact same app in Firestore, AWS Datastore, PouchDB, RxDB and WatermelonDB.
• Relational Databases Aren’t Dinosaurs, They're Sharks
• Partitioning GitHub’s relational databases to handle scale
• Real world database latency
• Write a time-series database engine from scratch
• The pedantic checklist for changing your data model in a web application
• Common data model mistakes made by startups
• An unlikely database migration
• If All You Have Is a Database, Everything Looks Like a Nail - Databases are fabulous resources. Of course, we’ll abuse them in ways that cause challenges.
• We deleted the production database by accident
• The database I wish I had (2020)
• What are databases? from Prisma's Data Guide.
• Recent database technology that should be on your radar (part 1)

Books

• Use The Index, Luke by Markus Winand - A site explaining SQL indexing to developers—no crap about administration.
• SQL Performance Explained by Markus Winand - Everything developers need to know about SQL performance.
• Designing Data-Intensive Applications (DDIA) - The big ideas behind reliable, scalable, and maintainable systems. (review, review, notes, literature references, Tweet)
• Database System Concepts

Theory and Research

CS theory and research.

Papers

• Closing the B-tree vs. LSM-tree Write Amplification Gap on Modern Storage Hardware with Built-in Transparent Compression by ScaleFlux et al., 2021, VLDB.
• LSI: A Learned Secondary Index Structure by MIT CSAIL et al., aiDM 2022.

Readings

• Readings in Database Systems (commonly known as the "Red Book") written by Peter Bailis, Joseph M. Hellerstein, Michael Stonebraker, editors
• database-system-readings is a curated reading list about database systems, including personal digests of books/papers/blogs.

Newsletters

• DB Weekly - A weekly round-up of database technology news and articles covering new developments, SQL, NoSQL, document databases, graph databases, and more.

Lists

• Database Systems Resources - This repo contains a list of resources that are hopefully useful for people who want to learn more about database systems. Resources include papers, university courses, blog posts, talks, and other sources.
• Awesome Database Tools - Everything that makes working with databases easier.
• Awesome Database Learning - A list of learning materials to understand databases internals, including but not limited to papers, blogs, courses, and talks.