By Louis Mckenna – BI Developer
Introduction
For the uninitiated, a BI Developer’s job is to manage the software within a company that stores and queries (for example a data warehouse); and develop suitable methods and structures within said software to allow a business to gain useful insights. And whilst this is largely my role here at synvert TCM (formerly Crimson Macaw) there are certain technologies or nuances within it that you may find interesting.
Overall Structure
At synvert TCM we operate via a squad structure i.e., staff separated into teams with a variety of member skill sets which allows each squad to function independently. This allows people the freedom to pursue internal projects outside of client work with a consistent group of people. This has certainly improved our development process. It also allows us to tailor approaches to what best suits the squad, rather than a one-size-fits-all! For example, my squad has a daily stand-up to ascertain the state of play across our clients and projects, which we adjust in terms of frequency or timing if we need to.
This doesn’t mean it’s just squads and nothing else though; we have weekly company-wide catch-ups to keep us aware of any interesting new developments or fun company news. The last couple of years has really highlighted to all of us how important social contact is. The downside of being split into a small, self-contained team (especially hyrid working) is that you lose opportunities to talk to people. To this end, we have casual work chat channels and lunchtime socials. We also have a weekly timeslot set aside for a drink and a chat to make sure we’re still connecting.
Technologies Used
Now for the fun stuff! My day-to-day work life typically revolves around Snowflake, dbt and Matillion. Snowflake is a cloud-based data warehousing service that we typically use to store client data and I mostly use it to help with client queries as well as manage access and security around said data. We also use this to build the tables and views that go into creating a client’s data warehouse in the first place.
Matillion and dbt are powerful and versatile ETL tools that can be used on top of Snowflake to transform raw data to something palatable and useful for clients, which excitingly doesn’t just extend to joins and calculations! I have also used them in the process of exporting data for clients. I’ve also used the, for importing and merging data from multiple sources.
That’s not all, however, here at synvert TCM, we preach the importance of t-shaped team members. This means that although you specialise in a particular field you have sufficient training to have a good understanding of other fields. It also offers the flexibility to transition to one of these fields if it suits you! As such although my day mainly involves working with interesting data in Snowflake or creating transformations in Matillion as well as dbt (check out Chris’ blog here about the value of macros in dbt) I can also be involved in visualisation work in Tableau, Looker and PowerBI. I can even get qualified on the AWS (Amazon Web Services) side of things. This is great for those who want to know how the data you’re manipulating actually gets to you!
Conclusion
This was just a quick look at a day in the life of a BI Developer at synvert TCM. Hopefully, you learned something new. Feel free to check out some of the other blogs on our website.
