Lesson 1 of 8 · 8 min · Working with tables in pandas

DataFrames and Series: loading and inspecting data

Covers: pandas 3.x. Updated 13 Nov 2026. Part of the Data Analyst roadmap.

What you will learn

  • What a DataFrame and a Series are
  • Load a CSV or Excel file and inspect it
  • Select columns and create new ones
  • Understand data types and the index
  • Save results to a file

In simple words

pandas is a Python tool for working with tables. A DataFrame is a table in your program: rows and columns with names, like an Excel sheet or a SQL table. A Series is just one column of it. You load a file into a DataFrame, look at it, and then ask questions of it in a line or two of code, instead of clicking through thousands of cells.

Words to know

  • DataFrame: a table of rows and columns in pandas.
  • Series: a single column of a DataFrame, with an index.
  • Index: the labels for the rows.
  • dtype: the data type of a column, such as integer, decimal, text or date.

Setup

python -m venv .venv
source .venv/bin/activate
pip install pandas openpyxl matplotlib

openpyxl lets pandas read and write .xlsx files. In your code:

import pandas as pd

This course uses pandas 3.x. Differences from 2.x that matter for beginners are noted where they appear: text columns have their own str type, and changes to a derived table never alter the original (copy-on-write).

Make a small DataFrame

Throughout this course we reuse the same sample, which mirrors the SQL course:

orders = pd.DataFrame({
    "order_id":    [101, 102, 103, 104, 105, 106],
    "customer_id": [1, 1, 2, 3, 3, 1],
    "order_date":  pd.to_datetime(["2025-09-01", "2025-09-15", "2025-09-20",
                                   "2025-10-02", "2025-10-10", "2025-10-20"]),
    "amount":      [1200, 800, 1500, 600, 900, 500],
    "status":      ["paid", "paid", "refunded", "paid", "pending", "paid"],
})

A dictionary maps each column name to its list of values. In real work you will load a file instead.

Load a file

orders = pd.read_csv("orders.csv", parse_dates=["order_date"])
sheet  = pd.read_excel("sales.xlsx", sheet_name="Oct")

Useful options of read_csv: sep=";" for another separator, encoding="utf-8" (or "latin-1" for old files), usecols=[...] to read only some columns, dtype={"pincode": "str"} to stop codes losing leading zeros, na_values=["NA", "-"] to mark extra missing-value symbols, and nrows=1000 to peek at a big file.

The five commands to run first

orders.head()          # first 5 rows (use .head(10) for more; .tail() for the last)
orders.shape           # (6, 5): rows, columns
orders.dtypes          # the type of each column
orders.info()          # columns, non-null counts, dtypes, memory
orders.describe()      # count, mean, std, min, quartiles, max of numeric columns

Sample results:

>>> orders.shape
(6, 5)
>>> orders.dtypes
order_id                int64
customer_id             int64
order_date    datetime64[us]
amount                  int64
status                    str
dtype: object

(The date shows datetime64[us] in pandas 3; older versions show [ns]. The text column shows str in 3.x and object in 2.x.) Read info() carefully: a number column that shows as text, or a date column that shows as text, is the first sign of dirty data. describe(include="all") also summarises text columns, and orders["status"].value_counts() counts each category:

status
paid        4
refunded    1
pending     1
Name: count, dtype: int64

Series: one column

orders["amount"]            # a Series (one column)
orders["amount"].sum()      # 5500
orders["amount"].mean()     # 916.666...
orders["amount"].max()      # 1500
orders["status"].unique()   # the distinct values
orders["customer_id"].nunique()   # how many distinct: 3

A Series has an index (the row labels, 0 to 5 here) and a dtype. Use two brackets for several columns, which returns a DataFrame:

orders[["order_id", "amount"]]

Create and change columns

New columns come from calculations on existing ones, applied to every row at once (no loop):

orders["amount_with_gst"] = orders["amount"] * 1.18
orders["is_paid"] = orders["status"] == "paid"
orders["month"] = orders["order_date"].dt.to_period("M")
orders["big_order"] = orders["amount"] >= 1000

.dt gives access to date parts (.dt.year, .dt.month, .dt.day_name()). Rename or drop:

orders = orders.rename(columns={"amount": "order_value"})
orders = orders.drop(columns=["is_paid"])
orders = orders.sort_values("order_value", ascending=False)

Notice the pattern: methods return a new DataFrame, and you assign it back. In pandas 3 this is the rule everywhere: derived tables are independent copies in behaviour (copy-on-write), so changing a subset never silently changes orders.

The index

Rows are identified by the index. The default is 0, 1, 2.... Make a column the index if it is a natural key:

by_id = orders.set_index("order_id")
by_id.loc[103]      # the row for order 103
by_id = by_id.reset_index()   # back to a normal column

After filtering or sorting the index labels stay with their rows (so you may see 3, 0, 5). Use reset_index(drop=True) for a fresh 0..n-1.

Save the result

orders.to_csv("orders_clean.csv", index=False)       # index=False avoids an extra column
orders.to_excel("orders_clean.xlsx", index=False)

Common mistakes

  • Not looking at the data first (head, dtypes, info).
  • Dates and numbers read as text, because of stray symbols; fix with parse_dates, dtype or conversion lessons later.
  • Looping over rows with for instead of whole-column operations, which is slow and long.
  • Saving with the index, adding an unwanted Unnamed: 0 column on reload.
  • Forgetting to assign the result of a method such as drop or sort_values.

Try it yourself

  1. Install pandas and recreate the orders DataFrame. Print shape, dtypes and describe().
  2. Add a column for GST at 18 percent and a flag for orders over 1000.
  3. Count orders per status with value_counts().
  4. Save the table to CSV without the index and read it back, checking that the dates are still dates (parse_dates).
  5. Load any CSV from your computer and write down three things info() tells you about it.

Check yourself

What is the difference between a DataFrame and a Series?

A DataFrame is a two-dimensional table with several columns. A Series is a single column with its index.

How do you select two columns?

With a list inside brackets: df[["a", "b"]], which returns a DataFrame.

Why use index=False in to_csv?

Otherwise pandas writes the row index as an extra first column, which shows up as Unnamed: 0 when you read the file again.

Which commands do you run first on a new dataset?

head(), shape, dtypes, info() and describe(), plus value_counts() on categories.

Interview questions

How do you add a calculated column in pandas?

Assign a calculation on existing columns to a new column name, such as df["total"] = df["price"] * df["qty"]; it is applied to all rows at once (vectorised).

Why avoid looping over DataFrame rows?

Vectorised operations run in optimised compiled code and are much faster and shorter than Python loops; iterrows is slow and rarely needed.

What does copy-on-write mean in pandas 3?

Any DataFrame or Series derived from another behaves as an independent copy, so modifying it never changes the original, and pandas copies data lazily only when needed.

Summary

  • A DataFrame is a table; a Series is one column; the index labels the rows.
  • Load with read_csv or read_excel, then inspect with head, shape, dtypes, info, describe.
  • Create columns with whole-column arithmetic; methods return new objects, so assign them back.
  • Save with to_csv(index=False).

Going deeper

See how much of this you can explain

A free mock interview asks you new questions on Python for data analysis with pandas for Data Analyst and scores every answer, with what to fix.