Sidecardata

The result
2 days
Revamped Outdated Backend
2 weeks
Deployed on AWS
1 month
Transitioned To Production
Technologies
Django
Django Rest Framework (DRF)
Next.js
ElactisCache
ECS
SQS
OpenAI
Slack
MS Teams

Overview

Sidecar Data is a platform designed to automate data model management for Data Engineers and business owners across various third-party platforms such as DbtCloud, Fivetran, and SnowFlake.

Problem Definition

Data Engineers and business owners often face challenges in manually managing data models, leading to inefficiencies and potential errors. The lack of automation in tasks such as monitoring data models on DBT Cloud and ensuring seamless integration with other platforms like SnowFlake and FiveTran can result in increased workload and decreased productivity. Additionally, the absence of standardized processes for identifying and addressing issues within data models can hinder the timely delivery of insights and actionable intelligence to stakeholders.

What we did

  • Checking failed tasks and generating reports.
  • Identifying models lacking test cases.
  • Monitoring resource connections between DbtCloud, SnowFlake, and FiveTran.
  • Integrating Large Language Models (LLM) to handle DBT Cloud data model issues automatically.
  • Automating notification and modification processes for businesses' data models.

Impacts

  • Facilitated the transformation of customer ideas into tangible, usable products.
  • Reduced manual tasks for Data Engineers, enhancing efficiency.
  • Created an open ecosystem capable of integrating with multiple third-party systems.
  • Addressed business challenges using data models from Dbt Cloud, including notifications, resource management, and software integration.

Explore our projects

See more
arrow-icon

We are ready to have a talk with you

Talk with our experts
arrow-icon