Open in app
Home
Notifications
Lists
Stories

Write
Aman Ranjan Verma
Aman Ranjan Verma

Home

About

Published in Towards Data Engineering

·Pinned

Become a Data Engineer

Roadmap towards becoming a Data Engineer — What do I do as a data engineer? I deal with moving, processing, and storing large volumes of data. I achieve it by writing jobs. It is also called the ETL pipeline. ETL stands for Extract Transform Load. I move data from multiple places, it can be a database, object store, or data warehouse. They are called sources…

Data Engineering

4 min read

Become a Data Engineer
Become a Data Engineer

Published in Towards Data Engineering

·May 29

SCD: Slowly Changing Dimension

A Data Warehousing Concept — A data warehouse is an OLAP tool to store and process a massive volume of structured data. One of the objectives is to store historical snapshots of the data coming from the OLTP systems. It means whatever append, update, and delete operations that are performed on the OLTP databases should…

Data Engineering

5 min read

SCD: Slowly Changing Dimension
SCD: Slowly Changing Dimension

Published in Towards Data Engineering

·May 22

File Formats I deal as a Data Engineer

JSON, XML, CSV, Parquet, ORC, and Avro e.t.c — As a data engineer, one of my tasks is to extract and load data. Data is not always extracted and stored in a database. Quite frequently you will have to face use cases where you need to deal with data stored in files. The most common file format in the…

Data Engineer

4 min read

File Formats I deal as a Data Engineer
File Formats I deal as a Data Engineer

Published in Analytics Vidhya

·May 20

Data Warehousing Schemas

Example: Star Schema, Snowflake Schema, Fact Constellation Schema — Much like an OLTP system(database), an OLAP system(data warehouse) is also required to maintain a schema. A database uses an entity relational model, while a data warehouse uses a dimensional model. In this blog, we will see the three major schemas that are used to model a data warehouse. About Star Schema Each…

Data Engineering

3 min read

Data Warehousing Schemas
Data Warehousing Schemas

Published in Towards Data Engineering

·May 10

Database Vs Data lake Vs Data warehouse

Understand technical differences and use cases for Interview purposes — In the Big-Data ecosystem or preparing for a data engineering interview, we often come across these keywords like databases, data lakes, and data warehouses. Since all these tools are used to hold data, this makes us think about what exactly are the key differences. In this blog, we will precisely…

Data Engineering

3 min read

Database Vs Data lake Vs Data warehouse
Database Vs Data lake Vs Data warehouse

Published in Towards Data Engineering

·Apr 29

SQL for everyday use

Top 25 most important SQL interview questions for the beginner — As IT professionals, be it, software developers, data engineers, data scientists, data analysts, or business analysts. Everyone is bound to have a working knowledge of SQL. In this blog, I have tried to capture the basic SQL queries that come in handy in everyday use. The blog contains examples on: …

Sql

9 min read

SQL for everyday use
SQL for everyday use

Apr 8

PySpark: Derive stats from your data

More than df.describe() — While working with data in PySpark, we often do data manipulation in the form of transformations. We receive data from stakeholders or some data pipelines. How do you make sure that the data that is sent to you is correct? …

Pyspark

2 min read

PySpark: Derive stats from your data
PySpark: Derive stats from your data

Published in Towards Data Engineering

·Apr 4

PySpark: Write data frame with the specific file name

Write PySpark data frame with specific file names in CSV/Parquet/JSON format — Spark users find it difficult to write files with a name of their choice. The default name that spark uses is the part files. In scenarios where we build a report or metadata file in CSV/JSON format, we want to save it with a specific name. There are some cases…

Data Engineering

2 min read

PySpark: Write data frame with the specific file name
PySpark: Write data frame with the specific file name

Published in Towards Data Engineering

·Jan 26

Data Engineering Interview Keywords: Part VI

Data Engineer Interview Questions — This is part six of the data engineer interview series. If you have not already, I highly recommend you to read Part I to build an understanding of my career background. Data Engineer Interview Questions: Part I Big Data Interview Questions Spark, SQL, Python, Data Modeling, and Data Warehouse, Data Structure & Algorithmmedium.com In this blog, I have discussed the keywords that you might encounter in data engineer interviews. …

Data Engineering

3 min read

Data Engineering Interview Keywords: Part VI
Data Engineering Interview Keywords: Part VI

Published in Towards Data Engineering

·Jan 26

Data Modeling Interview for Data Engineers: Part V

Flipkart Interview Experience — This is part five of the data engineer interview series. If you have not already, I highly recommend you to read Part I to build an understanding of my career background. Data Engineer Interview Questions: Part I Big Data Interview Questions Spark, SQL, Python, Data Modeling, and Data Warehouse, Data Structure & Algorithmmedium.com In this blog, I have discussed a data modeling interview question that was asked me while I was interviewing…

Data Engineering

3 min read

Data Modeling Interview for Data Engineers: Part V
Data Modeling Interview for Data Engineers: Part V
Aman Ranjan Verma

Aman Ranjan Verma

Data engineer @Flipkart, I post weekly. https://www.linkedin.com/in/ar-verma/

Following
  • Medium Creators

    Medium Creators

  • Netflix Technology Blog

    Netflix Technology Blog

  • Frank Andrade

    Frank Andrade

  • Barack Obama

    Barack Obama

  • Mikkel Dengsøe

    Mikkel Dengsøe

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Knowable