Seattle Data Guy

Are you trying to learn more about data engineering and MLOps this week?

Here are 7 great articles and post that can get you up to speed fast on these two topics.

1. Normalization Vs Denormalization - Taking A Step Back And Data Modeling
seattledataguy.substack.com/p/normalization-vs-den…

2. How And Why We Need To Implement Data Quality Now!
seattledataguy.substack.com/p/how-and-why-we-need-…

3. Multimodality and Large Multimodal Models (LMMs) by Chip Huyen
huyenchip.com/2023/10/10/multimodal.html

4. How Does League Of Legends Deploy Machine Learning Models Into The Game? by Ian Schweer
seattledataguy.substack.com/p/how-does-league-of-l…

5. The Realities Of Airflow - The Mistakes New Data Engineers Make Using Apache Airflow
https://www.youtube.com/watch?v=gkKY6...

6. Growing From Analyst To Data Engineer In 100 Days
seattledataguy.substack.com/p/growing-from-analyst…

7. What does an 𝗘𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗘𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁 look like? by Aurimas Griciūnas
www.linkedin.com/feed/update/urn:li:activity:71779…

Feel free to share some of your favorite posts below as well!

1 year ago | [YT] | 14

Seattle Data Guy

Data engineering isn't easy. It's not just about writing SQL queries and calling it a day.

There is...

- Constantly changing data infrastructure, like Hadoop, Flink, Snowflake and Databricks
- Data pipelines often break not because of there design but because the source systems changes without telling you
- Data governance and compliance - ensuring that data handling meets stringent legal and security standards.
- A crazy amount of orchestration and ETL tools exist
- Mastering query optimization, indexing strategies, and data partitioning to efficiently handle large volumes of data.
- Understanding multiple disciplines and technologies including networking, system architecture, SFTP, etc
- Trying to implement business logic to fix issues from the application

This barely scratches the surface.

I'm not saying this to complain by the way. I love data engineering.

If you'd like to learn more about data engineering and infra you can check out my newsletter here - seattledataguy.substack.com/

1 year ago | [YT] | 89

Seattle Data Guy

I created a guide for data engineers, software engineers, data scientists, analysts, and other technical professionals who want to start consulting.

It has templates for proposals, decks, and emails as well as links to books, YouTube channels, and videos you should look into if you want to start your own consulting company!


I'm accepting PRs so we can crowdsource this effort and have a really comprehensive list of resources!

Like, repost, share to spread the word and make this as awesome as possible for everyone.

github.com/sdg-1/consulting-handbook

1 year ago | [YT] | 150

Seattle Data Guy

How is your new job going?

1 year ago | [YT] | 221

Seattle Data Guy

When someone from the business team says, "I just need data xyz, it shouldn't be hard"

2 years ago | [YT] | 123

Seattle Data Guy

Do you want to start a data consulting company?

I am starting to do lives to help guide you so you don't make the same mistakes I made.

Sign up with the form below to not miss out as I will put it on a calendar invite.

forms.gle/v5tTCovNrJK5izQS8

2 years ago | [YT] | 14

Seattle Data Guy

Thank you all!

We have officially passed 50,000 subscribers!

I will be doing a live later in January. I have a few other events that I need to do first. But we will celebrate!

2 years ago | [YT] | 99

Seattle Data Guy

Do companies need to do better when it comes to making it clear what is expected at different levels to get promotions?

60% of the respondents to my 2022 State Of Data survey aren’t confident they know what is required to get a promotion. This made me re-examine what I know about leveling and skill expectations.
Seriously, what is expected at different levels?

I have found that tech companies will break down what is expected pretty clearly(even if it’s not always followed to a T ). Different levels are based on the axes they find valuable.

This could be based on axes such as the ones stated by Omar Halabieh (Head of Technology at Amazon Payment Services) which were:
- Independence
- Complexity
- Scope
- Ambiguity
- Influence
- Impact

Facebook also had a similar set of axes including impact, direction, scope, people, and engineering excellence. I would say tech companies are in the minority when it comes to having clear axes and expectations (based on discussions I have had).

Most people I have talked to find that their promotions in standard enterprise companies are somewhat attached to project deliverables, years of experience, and the hands of fate.

That aside, for this article, I will be focusing on breaking down the expectations at various levels for scope, influence, impact, complexity, and independence.

Read The Article Here -> seattledataguy.substack.com/p/what-does-it-take-to…

2 years ago | [YT] | 40

Seattle Data Guy

Are you planning to found a start-up in the next 2-5 years?

2 years ago | [YT] | 10

Seattle Data Guy

If I put together a conference should it be online or in person?

2 years ago | [YT] | 14