SDO 009 - Architecting Quality Data - Robert Harmon
scalingdataops.substack.com
What are your thoughts on data modeling? The current state of data is not pretty as we start to reach the limitations of the current iteration of the Modern Data Stack (MDS). With AWS being released in 2006, cloud computing ushered in a new era of data practices that were no longer constrained by the high investment of on-prem databases. Coupled with the hype of big data, data science (AKA the “sexiest job of the 21st century”), and startups burning cash to stoke network effects, speed became a higher priority than reliability and the MDS became the toolbox to enable this. With that said, this tradeoff brought hyper growth to the data industry and pushed the limit of what’s capable of data— but we are now experiencing growing pains that’s making it hard to navigate our data swamps. Once again, history is repeating itself as our industry looks back to the foundations of data modeling.
SDO 009 - Architecting Quality Data - Robert Harmon
SDO 009 - Architecting Quality Data - Robert…
SDO 009 - Architecting Quality Data - Robert Harmon
What are your thoughts on data modeling? The current state of data is not pretty as we start to reach the limitations of the current iteration of the Modern Data Stack (MDS). With AWS being released in 2006, cloud computing ushered in a new era of data practices that were no longer constrained by the high investment of on-prem databases. Coupled with the hype of big data, data science (AKA the “sexiest job of the 21st century”), and startups burning cash to stoke network effects, speed became a higher priority than reliability and the MDS became the toolbox to enable this. With that said, this tradeoff brought hyper growth to the data industry and pushed the limit of what’s capable of data— but we are now experiencing growing pains that’s making it hard to navigate our data swamps. Once again, history is repeating itself as our industry looks back to the foundations of data modeling.