What are the best resources to follow for Data Engineering?

Software Development
#1

For data engineers, there is currently no clear or formal path available. Most people in this position learned on the job rather than pursuing a specific path. My goal in writing this essay was to assist anyone who wants to become a data engineer but is unsure where to begin or where to locate study resources.

  • The Data Engineering Podcast: Tobias Macey’s weekly highlight of a new tool or creator has made a significant contribution to the data engineering community. The topics range from establishing web scale streaming services to creating organizational knowledge graphs to creating a more efficient data structure — but I always leave having learnt something new.

  • A Beginner’s Guide to Data Engineering: A popular piece about data engineering from an Airbnb data scientist. The author explains why data engineering is such an important feature of every machine learning project before delving into the many components of this subject. This is a must-read for all aspiring data engineers AND data scientists, in my opinion.

  • r/dataengineering on Reddit: Great articles and information frequently make their way into this forum, and you can pick up some pearls of wisdom from other comments. Although it’s a little more tranquil, it’s a terrific place to keep an eye on.