How to filter dataframe in Python?

Filtering DataFrames in Python: A Comprehensive Guide

Introduction

DataFrames are a powerful data structure in Python, providing a convenient way to manipulate and analyze large datasets. One of the most common tasks when working with DataFrames is filtering them to extract specific data. In this article, we will explore the various ways to filter DataFrames in Python, including using the loc and iloc methods, the query function, and the apply function.

Using the loc Method

The loc method is a powerful tool for filtering DataFrames. It allows you to select rows and columns based on specific conditions.

• Syntax: df.loc[row_index, column_name]
• Example:

  
  
  import pandas as pd

data = {‘Name’: [‘John’, ‘Anna’, ‘Peter’, ‘Linda’],
‘Age’: [28, 24, 35, 32],
‘Country’: [‘USA’, ‘UK’, ‘Australia’, ‘Germany’]}
df = pd.DataFrame(data)

filtered_df = df.loc[df[‘Age’] > 30]

print(filtered_df)

**Using the `iloc` Method**



The `iloc` method is similar to the `loc` method, but it allows you to select rows and columns based on integer positions.



*   **Syntax:** `df.iloc[row_index, column_name]`

*   **Example:**

    ```python

import pandas as pd



# Create a sample DataFrame

data = {'Name': ['John', 'Anna', 'Peter', 'Linda'],

        'Age': [28, 24, 35, 32],

        'Country': ['USA', 'UK', 'Australia', 'Germany']}

df = pd.DataFrame(data)



# Filter the DataFrame to get only the rows where Age is greater than 30

filtered_df = df.iloc[1:4, 0:2]



print(filtered_df)

Using the query Function

The query function is a convenient way to filter DataFrames. It allows you to select rows and columns based on specific conditions.

• Syntax: df.query('condition')
• Example:

  
  
  import pandas as pd

data = {‘Name’: [‘John’, ‘Anna’, ‘Peter’, ‘Linda’],
‘Age’: [28, 24, 35, 32],
‘Country’: [‘USA’, ‘UK’, ‘Australia’, ‘Germany’]}
df = pd.DataFrame(data)

filtered_df = df.query(‘Age > 30’)

print(filtered_df)

**Using the `apply` Function**



The `apply` function is a powerful tool for filtering DataFrames. It allows you to apply a function to each row or column of the DataFrame.



*   **Syntax:** `df.apply(func, axis=0)`, `df.apply(func, axis=1)`

*   **Example:**

    ```python

import pandas as pd



# Create a sample DataFrame

data = {'Name': ['John', 'Anna', 'Peter', 'Linda'],

        'Age': [28, 24, 35, 32],

        'Country': ['USA', 'UK', 'Australia', 'Germany']}

df = pd.DataFrame(data)



# Filter the DataFrame to get only the rows where Age is greater than 30

filtered_df = df.apply(lambda row: row['Age'] > 30, axis=1)



print(filtered_df)

Table: Filtering DataFrames

Method	Syntax	Example
loc	df.loc[row_index, column_name]	df.loc[df['Age'] > 30]
iloc	df.iloc[row_index, column_name]	df.iloc[1:4, 0:2]
query	df.query('condition')	df.query('Age > 30')
apply	df.apply(func, axis=0)	df.apply(lambda row: row['Age'] > 30, axis=1)

Conclusion

Filtering DataFrames is an essential step in data analysis and manipulation. The loc, iloc, query, and apply methods provide a range of options for selecting specific data from DataFrames. By mastering these methods, you can efficiently extract and manipulate data in your Python projects.

Additional Tips

• Always check the documentation for the specific method you are using to ensure you are using it correctly.
• Use the print function to inspect the filtered DataFrame and verify that the data is as expected.
• Consider using the reset_index method to reset the index of the DataFrame after filtering, which can simplify the process of selecting specific columns.

By following these guidelines and using the various methods provided in this article, you can effectively filter DataFrames in Python and extract the data you need for your projects.