Unifying an information stack and leveraging self-service analytics with Atlan
The Lively Metadata Pioneers collection options Atlan purchasers who’ve accomplished a complete evaluation of the Lively Metadata Administration market. Passing on what you’ve got realized to the subsequent knowledge chief is the true spirit of the Atlan neighborhood! That is why they’re right here to share their hard-earned insights into an evolving market, what makes up your trendy knowledge stack, revolutionary use circumstances for metadata, and extra.
On this installment of the collection, we meet Daniel Ferguson, director of information and analytics at PHMG, an audio branding firm that helps greater than 36,000 clients in 56 international locations have their finest sound. Daniel shares how PHMG reworked their knowledge stack from fragmented to unified, and the way Atlan has been a vital piece in monitoring lineage, managing reporting, and facilitating workforce onboarding.
This interview has been edited for brevity and readability.
May you inform us a little bit about your self, your background and what attracted you to Knowledge & Analytics?
I was a DJ after which labored as a sound engineer constructing recording studios. Throughout my time as a sound engineer, I took an interest within the technical and analytical facet of issues. After beginning a household, I needed a change. After promoting my recording studio, my mom, who ran a municipal workplace, supplied me a job and I began within the name heart unit, taking calls. It rapidly grew to become obvious that I may do extra than simply calls, so I moved to the database workforce.
I began learning a level in Economics and Mathematical Sciences at The Open College whereas engaged on the council. Utilizing the talents realized in faculty, I started to construct logistic regression fashions to focus on contacts on the name heart the place I had beforehand labored. I proposed that, with only one particular person, I may obtain the identical outcomes as the complete workforce. My work generated 300% extra outcomes than the workforce’s mixed efforts by streamlining knowledge assortment, addressing lacking data, and selecting the right contacts. After that, I received hooked on the facility of information and analytics.
I then arrange an organization offering knowledge providers to different native authorities. Close to the tip of my research, a consultancy in Scotland, Aquila Insights, supplied me a place. They labored with purchasers comparable to Sony, Workplace Depot and RBS, which gave me an early introduction to the info career. From there, I superior by way of the sphere and ultimately joined PHMG. My journey into knowledge was considerably unintended, however it led me to the place I’m at the moment.
Would you thoughts describing PHMG?
We focus on audio branding. Take into consideration logos like Netflix or Disney Plus – by way of sound alone, these manufacturers are immediately recognizable as leaders within the leisure and streaming trade, even when their visible logos aren’t on show.
We additionally transcend conventional audio branding by creating customized music tailor-made to every group. We’ll take Atlan for example: What’s Atlan about? What do you characterize? What’s the sort of rhythm you need to deliver?
This connection between music and id is what attracted me to the corporate. We’ve been very profitable, working in 56 international locations with 36,000 clients.
May you describe your knowledge stack and the way it got here collectively?
After I received right here, we used SQL servers with Excel spreadsheets. There have been little to no interactive experiences and every knowledge request needed to be introduced to the info workforce.
It was essential to modernize the data that flows to the corporate and implement the suitable know-how to realize this effectively and reliably. I centered on discovering know-how options that streamlined operations and diminished the necessity for extra engineers.
I used to be very cautious with know-how choice, avoiding options for the sake of it and never constructing from scratch. Whereas Azure Cloth gives a complete resolution, for instance, it is nonetheless new and comes with extra dangers, however it’s one thing I am keeping track of. It is essential to decide on the most effective instruments for the job and ensure they work nicely collectively. Investing in a seamless course of with these instruments permits you to begin sturdy and display worth rapidly, with room to evolve as you scale.
In my board pitch, I highlighted two important instruments: Atlan and ThoughtSpot. I defined that whereas we may get by with out them, they’d make a big distinction. I needed governance to be constructed into our processes and as a substitute of assigning knowledge stewards with out clear route, we offered actionable reporting and comprehensible knowledge. With correctly organized knowledge, governance turns into easy and Atlan streamlines this course of.
I chosen Snowflake for its robustness and affordable value, and Fivetran for its dependable pipeline efficiency, which successfully handles our knowledge integration wants.
Applied PowerBI for government reporting and ThoughtSpot for our self-service knowledge wants. I am an enormous fan of ThoughtSpot as a result of it permits customers to fine-tune their very own experiences, decreasing the necessity for fixed modifications by the info workforce.
For orchestration, I exploit Airflow to handle pipelines and DBT with GitLab for our code repository and CI/CD processes.
Why was Atlan a very good match? Did something stand out throughout your analysis course of?
At my earlier group, I attempted to go open supply with DataHub, however it required important funding to keep up and develop. Atlan stood out as a result of it’s plug-and-play, robotically creating miners that reveal beforehand unknown data. It identifies and explains scripts that we did not learn about, saving time and decreasing technical debt from having to manually evaluate in depth code.
Atlan permits us to trace and monitor what now we have created, together with knowledge lineage and property. It is invaluable for reviewing experiences without having to ask for code particulars; Merely browse Atlan to view the report historical past. New workforce members can even perceive constructing experiences by way of Atlan.
For me, Atlan was a key piece of the puzzle.
I extensively researched Collibra, Alation, and Atlan, and Atlan was the clear alternative. It appeared designed for medium-sized corporations and required minimal engineering effort. Given our state of affairs, it was essential to combine it from the start, and never as an afterthought. This allowed us to be taught and develop Atlan alongside our current techniques, fairly than attempting to include it into our pre-built configuration.
I all the time make it some extent to fulfill with management groups at occasions to gauge their angle and dedication, and I do not know of some other participant who’s doing in addition to Atlan. I used to be genuinely impressed with Atlan’s management workforce: not solely their ardour for the product but additionally their dedication to addressing my challenges and enhancing our state of affairs.
How do you intend to leverage Atlan to enhance your knowledge stack? What fascinating use circumstances and targets do you keep in mind?
We’ve invested in an information vault mannequin for our knowledge warehouse, which feeds an operational knowledge warehouse, what I name the info mart. All our experiences and metrics are created from this knowledge mart. At Atlan, we outline construct all the pieces, so as soon as a metric is outlined, we are able to write the SQL to extract it from the market.
We then created curated tables for customer support and gross sales organizations, permitting them to self-service by way of ThoughtSpot. To acquire detailed details about the development and rationale of those metrics, we retailer that data in Atlan, which turns into our catalog.
As new individuals be a part of, I be certain there isn’t any want for a handover. By default, we doc our processes as we go and construct techniques that depart clear navigation paths for others to comply with. Atlan performs a vital function on this. We direct new workforce members to Atlan to assist them perceive how all the pieces is constructed and what it is constructed from. Atlan not solely reveals the code, however highlights key objects, their use and their significance.
One other necessary mission entails making a complete glossary inside Atlan, to function our single supply of reality. This setting permits enterprise customers to entry all company metrics and consider experiences from Salesforce, PowerBI, and ThoughtSpot, all linked round key KPIs.
We’re additionally at the moment refining our knowledge lineage and mannequin descriptions. As we create new knowledge fashions, we replace descriptions incrementally fairly than in bulk. This ongoing effort helps be certain that our knowledge fashions are well-documented and simply comprehensible.
Do you’ve gotten any tricks to share along with your friends who’re beginning to handle and set up their knowledge property successfully?
Firms all the time speak about being data-driven, however they do not speak concerning the property that truly drive the info. We would like data to circulate in our group, however data can’t circulate if it isn’t organized constantly. And for me, instruments like Atlan make it a lot simpler for us to prepare and talk necessary knowledge.
Do not get me fallacious, Atlan just isn’t a miracle resolution. It will not repair poor group inside your knowledge warehouse. Nevertheless, it gives a centralized place to outline and consider your processes, serving to you determine which of them are efficient and which of them want enchancment.
Atlan helped us decide the place to start out by figuring out our most vital boards and specializing in what was necessary. For instance, we discovered a foundational desk for all the pieces we constructed, which allowed us to prioritize it. We then evaluated our experiences and found that some we thought have been necessary have been related solely to particular experiences, to not the broader context.
As we get delicate knowledge, we are able to additionally flag it instantly. If we get audited, we are able to simply open Atlan and say, “Hey, that is what now we have. That is how we handle our knowledge. “These are our knowledge property.” Subsequently, these dedicated to utilizing knowledge should deal with their knowledge property and perceive what they’re.
Photograph by Adi Goldstein in unpack
