A Day in the Life of a BI Developer

By Louis Mckenna – BI Developer

Introduction

For the uninitiated, a BI Developer’s job is to manage the software within a company that stores and queries (for example a data warehouse); and develop suitable methods and structures within said software to allow a business to gain useful insights.  And whilst this is largely my role here at Crimson Macaw there are certain technologies or nuances within it that you may find interesting.

Overall Structure

At Crimson Macaw we operate via a squad structure i.e., staff separated into teams with enough variety in the skillsets of the members to allow each squad to function independently of one another.  This allows people the freedom to pursue internal projects outside of client work with a consistent group of people, which has certainly improved our development process. It also allows us to tailor approaches to what best suits the squad, rather than have a one-size fits all policy for everyone within the company!  For example, my squad has a daily stand up to ascertain the state of play across our clients and projects, which could be adjusted in terms frequency or timing if we needed it to be.

This doesn’t mean it’s just squads and nothing else though; we have weekly company-wide catch ups to keep us aware of any interesting new developments or fun company news.  As the last couple of years have really highlighted to all of us how important social contact is, the downside of being split into a small, self-contained team (especially when working from home) is that you lose opportunities to talk to people outside of it. To this end we have casual work chat channels and lunch-time socials as well as a weekly timeslot set aside for a drink and a chat to make sure we’re still connected.

Technologies Used

Now for the fun stuff!  My day-to-day work life typically revolves around Snowflake, dbt and Matillion.  Snowflake is a cloud-based data warehousing service that we typically use to store client data and I mostly use to help with client queries as well as manage access and security around said data. It’s also used to build the tables and views that go into to creating a client’s data warehouse in the first place.

Matillion and dbt are powerful and versatile ETL tools that can be used on top of Snowflake to transform raw data to something palatable and useful for clients, which excitingly doesn’t just extend to joins and calculations!  I have also used them in the process of exporting data for clients as well as importing and merging data from multiple sources.

That’s not all however, here at Crimson we preach the importance of t-shaped team members, meaning that although you specialise in a particular field you are trained up well enough to have a good understanding of the fields around you with the flexibility in place to transition to one of these fields if it suits you!  As such although my day mainly involves working with interesting data in Snowflake or creating transformations in Matillion as well as dbt (check out Chris’ blog here about the value of macros in dbt) I can also be involved in visualisation work in: Tableau, Looker and PowerBI.  I can even get qualified on the AWS (Amazon Web Services) side on things which is great for those who want to know how the data you’re manipulating actually gets to you!

Conclusion

This was just a quick look at a day in the life a BI Developer at Crimson Macaw, hopefully you learned something new. Feel free to check out some of the other blogs on our website.