KinShip
Pet Care Data Engineering Solutions for Kinship's Canine Health Platform
2019 - New York, USA
Pet Care Services
51-200 employees
Overview
Kinship, a division of Mars Petcare, is a leader in pet care technology, leveraging data to enhance pet health insights. Their Pet Insight platform collects vast volumes of activity data from IoT sensors embedded in pet collars, transforming raw information into actionable health insights. As data demands grew, Kinship partnered with Folio3, a trusted technology partner, to re-engineer its data infrastructure, focusing on advanced pet care data engineering solutions. This collaboration optimized data workflows and processing capabilities, enabling faster and more accurate health insights while laying the foundation for continued innovation in pet healthcare.
The Challenge – Efficiently Processing and Scaling Pet Data
Kinship's goal was to enhance its Pet Insight platform by efficiently processing large datasets from IoT sensors, which tracked canine activity over extended time frames. However, the system faced significant obstacles in its pet health data pipeline:
Scalability Bottlenecks: Processing data for thousands of dogs over months and years overwhelmed the existing infrastructure, hindering scalable pet data analytics solutions.
Slow Data Retrieval: Data older than 90 days took excessive time to fetch, limiting the ability to perform timely analyses.
Data Duplication Risks: The system lacked a reliable mechanism to prevent data duplication, leading to inefficient storage and skewed data integrity.
Inconsistent Data Processing: The system struggled to convert raw sensor data into usable formats, such as PetInsightTimeData (PITD) objects, slowing the flow of actionable information to machine learning models with the pet health data pipeline.
The Solution: Cloud-Driven Data Optimization To View Real-Time Pet Health Insights
Folio3 conducted a thorough audit of Kinship’s existing infrastructure, pinpointing critical inefficiencies and scalability limitations. The outcome was a robust pet care data engineering solution, leveraging PySpark on Databricks to optimize data processing workflows and enhance overall performance:
Technologies Involved In This Case
Amazon S3
DynamoDB
Databricks
Delta Lake
PySpark
Pandas
PySpark
Results & Achievements
Rapid Data Retrieval
With the newly implemented solution, data retrieval times were reduced from hours to minutes, allowing Kinship to access historical records faster than ever before and enabling quicker pet data analytics solutions.
Scalable Architecture
The introduction of horizontal auto-scaling ensured Kinship’s platform could effortlessly scale to handle increasing data volumes from thousands of animals, future-proofing the system for continued growth.
Elimination of Data Duplication
By preventing data duplication, Folio3 optimized storage usage and ensured clean, accurate data for Kinship’s machine learning models.
Accelerated Time-to-Insight
The revamped pet health data pipeline enabled Kinship’s data scientists to process and analyze large datasets in real-time pet health data, delivering faster, more accurate insights for improved pet care.
