[dijksterhuis]

Depends who you ask. Traditionally speaking:

# Data lake

Data is stored en masse with no schema applied, either unstructed or structured data can be dumped straight into the lake or can be transformed and then dumped in. Turns into a data swamp when it becomes unusable due to staleness or complexity.

Data lakes are basically an AWS S3 bucket business users can access and (attempt) to do reporting on.

# Data warehouse

Heavily structured schema applied to data used in reporting, usually defines the single point of truth for business purposes. Uses a star schema model (if you follow Kimball [0] methodology) to create dimension tables used to filter and aggregate raw measurements from the central fact tables (which contain your actual measures like £ made on 1 sale).

Kimball and Inmon [1] philosophies come with their own benefits and trade offs. See bottom of [2].

Edit: got methodolgies the wrong way round with initial costs, linked article has a useful table that I didn't see.

Data warehouses have a very concrete definition and are usually implemented via Kimball's or Inmon's method. When I've worked with them they've become the bastion of business reporting (excel users love a pivot table).

---

Just to confuse matters, there's also the data vault: https://en.m.wikipedia.org/wiki/Data_vault_modeling

0: https://en.m.wikipedia.org/wiki/Ralph_Kimball

1: https://en.m.wikipedia.org/wiki/Bill_Inmon

2: https://www.zentut.com/data-warehouse/kimball-and-inmon-data...

[laichzeit0]

Disagree with the swamp part. See it as a staging area for the warehouse and data science. If you use your data lake as the source for your warehouse then you’re forced to keep it clean. Also, for data science use cases you need access to the raw data and the what’s in the warehouse, much easier if the data lake can be used as opposed to essentially building one anyway.

[dijksterhuis]

I have 100% seen data swamps in the wild:

> We see customers creating big data graveyards, dumping everything into Hadoop distributed file system (HDFS) and hoping to do something with it down the road. But then they just lose track of what’s there. The main challenge is not creating a data lake, but taking advantage of the opportunities it presents.

https://en.wikipedia.org/wiki/Data_lake

> If you use your data lake as the source for your warehouse then you’re forced to keep it clean

So you apply a schema...? Determining what should and should not live in your storage? What format it should be in etc.?

That's not a data lake in my mind -- it's just the ETL staging area.

This is why I dislike the term data lake. It's so vague and loose that it could mean 300 different things to 300 different experts. Makes it a great term for snake oil marketing though.