Music royalty collection and calculation for 2 different countries.
Two independent digital music rights management organisations, one in Europe and the other in APAC, requested Klarrio to help scale and optimise their data pipeline. Klarrioās expertise and experience was instrumental in creating a scalable solution for the ever increasing volumes of data that needed to be ingested and processed.
Challenge
The increasing variation in music consumption, including the per-song instead of per-album consumption,Ā requires music rights organizations to use a scalable architecture to make sure the necessary analysis and calculations can be made so artists are compensated correctly for their work.Ā
Solution
The European organisation needed help to optimise the data ingestion capabilities of their platform. Klarrio analysed the performance bottlenecks, resolved them and reworked the ingestion pipeline to increase the scalability and performance.
The result was a massively scalable data ingestion and processing pipeline which is future proof to handle the ever growing volume of data.
For APAC, Klarrio rewrote a core part of the music royalties organisationās legacy systems to improve the speed in matching records from Apple, Google, Spotify, YouTube, and othersā music files by up to 50x, from days to hours/minutes. This resulted in scalability of the matching pipeline to cater for future needs and a reduced cost of implementation by removing manual processes. In addition, by moving to a public cloud infrastructure it helped facilitate and automate visualisation of key metrics, allowing business users to control the workflow andĀ interact with the system.
The Technology behind
- Scala
- Apache Spark
- Hadoop
- Elasticsearch
- Kafka
- Docker
- Kubernetes
”āThings work out best for those who make the best of how things work out.ā
ā John Wooden
Results
Speed | significant reductionĀ (up to 93%) of ingestion and processing execution time.Ā
Scale | scalable data model and architecture to meet future needs.
Optimize | Testing and performance tuning for an optimal performance
Micro-services | move from monolithic architecture to loosely coupled one.
Modernisation | Cobol and Java codes re-written in Scala.
Ask for a demo!
We’re your one-stop cloud-native partner
Big Data engineering. Data science. Data Analytics. Site Reliability Engineering. Consulting. And customized Open Source projects for companies of all sizes. Learn more about what Klarrio can do for you today.
Other Projects
Just a few projects examples.