AI & Data

Big Data Platform Design Using Apache Tools

The Big Data Platform Design Using Apache Tools training is a practical, 3-day workshop where participants will learn modern methods of building scalable and efficient Big Data platforms.

Duration
6h
Ask about the program → See all
Who it's for

Ideal for teams that…

1 IT specialists, Big Data architects, and data engineers aiming to design modern, scalable Big Data platforms
2 DevOps and administrators responsible for deploying and managing Hadoop/Spark/Kafka infrastructure
3 Data analysts and engineers who wish to understand the architecture and tools of Apache for data processing and analysis
4 Individuals planning to expand existing solutions or start new Big Data projects
Outcomes after the program

Hands-on AI and data analytics workshops — built around your team's real cases.

✓

Design and implement data pipelines for batch and stream processing

✓

Understand the principles of building modern, scalable Big Data architecture using Apache tools

✓

Gain skills in configuring and managing systems like Hadoop, Kafka, NiFi, Spark, and Flink

✓

Master techniques for managing metadata, data lineage, and automating workflows

✓

Learn best deployment practices and methods for optimizing and monitoring Big Data platforms

Program · 3 modules

What we actually do

M01
Day 1: Fundamentals of Big Data Architecture and Apache Tools
  • · Basic concepts and layers of Big Data architecture: data, processing, management, analysis
  • · Architecture models: Data Lake, Lambda, Kappa, Data Lakehouse
  • · Design criteria: data type, scalability, batch vs. stream processing
  • · Overview of data processing methods: batch vs. stream
  • · HDFS architecture: NameNode and DataNode roles
  • · Batch processing with MapReduce – basics and use cases
  • · Administration and monitoring of Hadoop clusters
  • · Functional programming concepts and Python vs. Java comparison
  • · Python elements for data processing: DataFrames, lambdas, comprehensions, map, filter
  • · Practical exercises: simple data processing and integration with Big Data tools (e.g. PySpark)
M02
Day 2: Data Processing and Integration Tools
  • · Apache Kafka architecture: producers, consumers, partitions, replication
  • · Apache NiFi: managing data flows and integrating sources and sinks
  • · Practical exercises: creating and monitoring data flows
  • · Spark architecture: RDD, DataFrame, Spark SQL
  • · Flink: stream processing, time windows, state management
  • · Designing batch and streaming jobs, optimization, Catalyst
  • · Integration with Apache Hadoop and application deployment
M03
Day 3 (Optional): Data Storage, Workflow Management, and Governance
  • · Apache Iceberg: scalable table format, ACID support, query optimization
  • · Apache Atlas: metadata management, governance, data lineage
  • · Apache Druid: architecture, indexing, real-time and batch analytics
  • · Designing workflows and managing dependencies with Airflow
  • · Implementing data pipelines and automating processing
  • · Integration with CI/CD tools and production environments
  • · Defining DAGs and working with tasks in Python and Bash
Every module is adapted to your stack and context. The above is a starting point — not a fixed agenda.
How we work

From brief to retro in 30 days.

01

Brief & diagnosis

A call with the team lead + a short survey for participants. We define goals, gap and context.

02

Program customization

We adapt modules, case studies and code examples to your stack. Approval in 5 days.

03

Workshop

Trainer-led sessions, hands-on, code review. Mentor available between sessions too.

04

Retro + report

Outcome report for the team and lead. 30 days of consulting included.

Inquiry

Send a brief. We'll reply within 1 day.

After a short brief we'll prepare a program and a quote. No obligations — it's just a starting point.

✓Quote within 48h of the brief
✓First session within 30 days
✓Pilot before the full decision
✓VAT invoice, payment in instalments possible

Ochrona antyspamowa (Cloudflare Turnstile) zostanie aktywowana po wpięciu klucza.

Other programs for teams

See all →
AI & Data

Active Directory Training

Hands-on AI and data analytics workshops — built around your team's real cases.

AI & Data →
AI & Data

Advanced Power BI Training

Hands-on AI and data analytics workshops — built around your team's real cases.

AI & Data →
AI & Data

Advanced RPA Developer Training

Hands-on AI and data analytics workshops — built around your team's real cases.

AI & Data →