Building Scalable Data Architecture for IoT with AFT-Managed Landing Zones

Challenges

The client initiated an AWS migration strategy with the goal of finding a dependable partner to manage cloud infrastructure, security, compliance audits, monitoring, logging, and CI/CD.

To optimize operations built around wearable IoT data, the client required:

• A secure and cost-optimized Data Lake for raw IoT data.
• A robust Data Warehouse for processed data.
• Scalable ETL pipelines to support data processing and analytics.
• Selection and implementation of a streaming platform for real-time processing.

Case study

Dmytro Petlichenko

5 min to read

Also read

Optimized Data Architecture: Transforming Data Costs into Growth

High Tech Software Cloud and Infrastructure Security Cloud Modernization Data Architecture Managed Services

Our Approach

To meet the client’s goals, Dedicatted built a secure, scalable AWS-native platform leveraging modern infrastructure and data processing tools.

Data Architecture

• AWS Lake Formation: Foundation for the Data Lake, supporting raw (images, sound files, CSV) and processed data.
• AWS MSK (Kafka): Primary streaming platform for ingesting IoT data.
• Managed Service for Apache Flink: For real-time stream data processing.
• AWS Glue: Used to build the data catalog.
• AWS Athena: Enabled serverless analytics on structured data.

DevOps & Infrastructure Automation

• Terraform: Infrastructure provisioning as code.
• AWS CodePipeline & AWS CodeBuild: CI/CD pipeline implementation.
• AWS Control Tower & Landing Zones: Centralized AWS account management.
• AWS AFT (Account Factory for Terraform): Enabled automated, scalable Control Tower management.
• AWS EKS & ArgoCD: For managed Kubernetes and GitOps deployment workflows.

Outcome

Dedicatted assumed full ownership of the client’s AWS infrastructure, implementing AWS Control Tower according to industry best practices and aligning the platform with both security and scalability goals.

Key achievements:

• Fully operational AWS Data Lake built with Lake Formation and integrated into ETL pipelines within 3 months.
• Clear separation of raw and processed data via isolated Data Lakes and Data Ponds, with experiment-level access controls.
• Provisioned real-time ETL pipelines using AWS MSK and Apache Flink.
• Automated CI/CD pipelines integrated with infrastructure and data processing services.
• AWS AFT enabled scalable management of over 20 AWS accounts.
• Full automation of account creation with GuardDuty SCP policies applied at the organization level.

The outcome of this collaboration is a secure, automated, and scalable AWS environment built from the ground up to support the client’s IoT data platform and long-term growth.