Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

DAMA-DMBOK2 covers this very comprehensively

https://www.dama.org/cpages/body-of-knowledge



Data engineering is cool and new while data management is old school and enterprise.

Specifically, data engineering in some tech companies is truly a revenue driver, so it makes data engineering in other organizations be viewed as a cost center so much, even if it is the same work at most organizations.


This may be nitpicking, but technologies being described as "cool" versus "enterprise" or "new" versus "old" I find meaningless. I don't necessarily want to have the "coolest" or "newest" tech stack; I want to have the tech stack that solves reasonably and reliably solves my business problems. If that means leveraging "old" or "enterprise" technologies and practices, that could be totally fine.


How do you define the two terms?


Data Engineering is an engineering displine -- it can involve anything from data ingestion, transformation, storage, enrichment, aggregation up to presentation in operational reports. But it's still a manufacturing process with "data" as an input and "data" as its output.

Data Management is an organization discipline -- it is about how the enterprise manages data as an asset and how data is embedded in the organization. This includes data governance issues like common data models, and a chain of command (which person/role is responsible for which piece of data), but also second-tier data processes such as quality control and data valuation.


Data engineering versus data management?

Data engineering is nominally more pipeline oriented and less concerned with the governance & people side of things, but good data engineering people end up driving a lot of data management work because that's what makes the data engineering less painful (eliminate root cause of data errors and annoying data requests) and data overall more useful and valuable.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: