Python for Beginners: Check for NaN Values in Pandas Python :
by:
blow post content copied from Planet Python
click here to view original post
While working with data in python, we often encounter null values or NaN values. In this article, we will discuss different ways to check for nan values or null values in a pandas dataframe or series.
- The isna() Function
- Check for NaN Values in a Pandas Dataframe Using The isna() Method
- Check for Nan Values in a Column in Pandas Dataframe
- Check for Nan Values in a Pandas Series Using The isna() Method
- Check for NaN Values in Pandas Using the isnull() Method
- Check for NaN Values in a Dataframe Using the isnull() Method
- Check for NaN in a Column in a Dataframe Using the isnull() Method
- Conclusion
The isna() Function
The isna() function in pandas is used to check for NaN values. It has the following syntax.
pandas.isna(object)Here, the object can be a single python object or a list/array of python objects.
If we pass a single python object to the isna() method as an input argument, it returns True if the python object is None, pd.NA or np.NaN object. You can observe this in the following example.
import pandas as pd
import numpy as np
x=pd.NA
print("The value is:",x)
output=pd.isna(x)
print("Is the value Null:",output)Output:
The value is: <NA>
Is the value Null: TrueIn the above example, we have passed the pandas.NA object to the isna() function. After execution, the function returns True.
When we pass a list or numpy array of elements to the isna() function, the isna() function is executed with each element of the array.
After execution, it returns a list or array containing True and False values. The False values of the output array correspond to all the values that are not NA, NaN, or None at the same position in the input list or array. The True values in the output array correspond to all the NA, NaN, or None values at the same position in the input list or array. You can observe this in the following example.
import pandas as pd
import numpy as np
x=[1,2,pd.NA,4,5,None, 6,7,np.nan]
print("The values are:",x)
output=pd.isna(x)
print("Are the values Null:",output)Output:
The values are: [1, 2, <NA>, 4, 5, None, 6, 7, nan]
Are the values Null: [False False True False False True False False True]In this example, we have passed a list containing 9 elements to the isna() function. After execution, the isna() method returns a list of 9 boolean values. Each element in the output list is associated with the element at the same index in the input list given to the isna() function. At the indices where the input list contains Null values, the output list contains True. Similarly, at indices where the input list contains integers, the output list contains False.
Check for NaN Values in a Pandas Dataframe Using The isna() Method
Along with the isna() function, the pandas module also has the isna() method at the dataframe level. You can directly invoke the isna() method on the pandas dataframe to check for nan values.
The isna() method, when invoked on a pandas dataframe, returns another dataframe containing True and False values. You can observe this in the following example.
import pandas as pd
import numpy as np
df=pd.read_csv("grade.csv")
print("The dataframe is:")
print(df)
output=df.isna()
print("Are the values Null:")
print(output)Output:
The dataframe is:
Class Roll Name Marks Grade
0 1 11 Aditya 85.0 A
1 1 12 Chris NaN A
2 1 14 Sam 75.0 B
3 1 15 Harry NaN NaN
4 2 22 Tom 73.0 B
5 2 15 Golu 79.0 B
6 2 27 Harsh 55.0 C
7 2 23 Clara NaN B
8 3 34 Amy 88.0 A
9 3 15 Prashant NaN B
10 3 27 Aditya 55.0 C
11 3 23 Radheshyam NaN NaN
Are the values Null:
Class Roll Name Marks Grade
0 False False False False False
1 False False False True False
2 False False False False False
3 False False False True True
4 False False False False False
5 False False False False False
6 False False False False False
7 False False False True False
8 False False False False False
9 False False False True False
10 False False False False False
11 False False False True TrueIn the above example, we have passed a dataframe containing NaN values along with other values. The isna() method returns a dataframe containing boolean values. Here, False values of the output dataframe correspond to all the values that are not NA, NaN, or None at the same position in the input dataframe. The True values in the output dataframe correspond to all the NA, NaN, or None values at the same position in the input dataframe.
Check for Nan Values in a Column in Pandas Dataframe
Instead of the entire dataframe, you can also check for nan values in a column of a pandas dataframe. For this, you just need to invoke the isna() method on the particular column as shown below.
import pandas as pd
import numpy as np
df=pd.read_csv("grade.csv")
print("The dataframe column is:")
print(df["Marks"])
output=df["Marks"].isna()
print("Are the values Null:")
print(output)Output:
The dataframe column is:
0 85.0
1 NaN
2 75.0
3 NaN
4 73.0
5 79.0
6 55.0
7 NaN
8 88.0
9 NaN
10 55.0
11 NaN
Name: Marks, dtype: float64
Are the values Null:
0 False
1 True
2 False
3 True
4 False
5 False
6 False
7 True
8 False
9 True
10 False
11 True
Name: Marks, dtype: boolCheck for Nan Values in a Pandas Series Using The isna() Method
Like a dataframe, we can also invoke the isna() method on a Series object in pandas. In this case, the isna() method returns a Series containing True and False values. You can observe this in the following example.
import pandas as pd
import numpy as np
x=pd.Series([1,2,pd.NA,4,5,None, 6,7,np.nan])
print("The series is:")
print(x)
output=pd.isna(x)
print("Are the values Null:")
print(output)Output:
The series is:
0 1
1 2
2 <NA>
3 4
4 5
5 None
6 6
7 7
8 NaN
dtype: object
Are the values Null:
0 False
1 False
2 True
3 False
4 False
5 True
6 False
7 False
8 True
dtype: boolIn this example, we have invoked the isna() method on a pandas series. The isna() method returns a Series of boolean values after execution. Here, False values of the output series correspond to all the values that are not NA, NaN, or None at the same position in the input series. The True values in the output series correspond to all the NA, NaN, or None values at the same position in the input series.
Check for NaN Values in Pandas Using the isnull() Method
The isnull() function is an alias of the isna() function. Hence, it works exactly the same as the isna() function.
When we pass a NaN value, pandas.NA value, pandas.NaT value, or None object to the isnull() function, it returns True.
import pandas as pd
import numpy as np
x=pd.NA
print("The value is:",x)
output=pd.isnull(x)
print("Is the value Null:",output)Output:
The value is: <NA>
Is the value Null: TrueIn the above example, we have passed pandas.NA value to the isnull() function. Hence, it returns True.
When we pass any other python object to the isnull() function, it returns False as shown below.
import pandas as pd
import numpy as np
x=1117
print("The value is:",x)
output=pd.isnull(x)
print("Is the value Null:",output)Output:
The value is: 1117
Is the value Null: FalseIn this example, we passed the value 1117 to the isnull() function. Hence, it returns False showing that the value is not a null value.
When we pass a list or numpy array to the isnull() function, it returns a numpy array containing True and False values. You can observe this in the following example.
import pandas as pd
import numpy as np
x=[1,2,pd.NA,4,5,None, 6,7,np.nan]
print("The values are:",x)
output=pd.isnull(x)
print("Are the values Null:",output)Output:
The values are: [1, 2, <NA>, 4, 5, None, 6, 7, nan]
Are the values Null: [False False True False False True False False True]In this example, we have passed a list to the isnull() function. After execution, the isnull() function returns a list of boolean values. Each element in the output list is associated with the element at the same index in the input list given to the isnull() function. At the indices where the input list contains Null values, the output list contains True. Similarly, at indices where the input list contains integers, the output list contains False.
Check for NaN Values in a Dataframe Using the isnull() Method
You can also invoke the isnull() method on a pandas dataframe to check for nan values as shown below.
import pandas as pd
import numpy as np
df=pd.read_csv("grade.csv")
print("The dataframe is:")
print(df)
output=df.isnull()
print("Are the values Null:")
print(output)Output:
The dataframe is:
Class Roll Name Marks Grade
0 1 11 Aditya 85.0 A
1 1 12 Chris NaN A
2 1 14 Sam 75.0 B
3 1 15 Harry NaN NaN
4 2 22 Tom 73.0 B
5 2 15 Golu 79.0 B
6 2 27 Harsh 55.0 C
7 2 23 Clara NaN B
8 3 34 Amy 88.0 A
9 3 15 Prashant NaN B
10 3 27 Aditya 55.0 C
11 3 23 Radheshyam NaN NaN
Are the values Null:
Class Roll Name Marks Grade
0 False False False False False
1 False False False True False
2 False False False False False
3 False False False True True
4 False False False False False
5 False False False False False
6 False False False False False
7 False False False True False
8 False False False False False
9 False False False True False
10 False False False False False
11 False False False True TrueIn the output, you can observe that the isnull() method behaves in exactly the same manner as the isna() method.
Check for NaN in a Column in a Dataframe Using the isnull() Method
Instead of the entire dataframe, you can also use the isnull() method to check for nan values in a column as shown in the following example.
import pandas as pd
import numpy as np
df=pd.read_csv("grade.csv")
print("The dataframe column is:")
print(df["Marks"])
output=df["Marks"].isnull()
print("Are the values Null:")
print(output)Output:
The dataframe column is:
0 85.0
1 NaN
2 75.0
3 NaN
4 73.0
5 79.0
6 55.0
7 NaN
8 88.0
9 NaN
10 55.0
11 NaN
Name: Marks, dtype: float64
Are the values Null:
0 False
1 True
2 False
3 True
4 False
5 False
6 False
7 True
8 False
9 True
10 False
11 True
Name: Marks, dtype: boolIn a similar manner, you can invoke the isnull() method on a pandas series as shown below.
import pandas as pd
import numpy as np
x=pd.Series([1,2,pd.NA,4,5,None, 6,7,np.nan])
print("The series is:")
print(x)
output=pd.isnull(x)
print("Are the values Null:")
print(output)Output:
The series is:
0 1
1 2
2 <NA>
3 4
4 5
5 None
6 6
7 7
8 NaN
dtype: object
Are the values Null:
0 False
1 False
2 True
3 False
4 False
5 True
6 False
7 False
8 True
dtype: boolIn the above example, we have invoked the isnull() method on a series. The isnull() method returns a Series of boolean values after execution. Here, False values of the output series correspond to all the values that are not NA, NaN, or None at the same position in the input series. The True values in the output series correspond to all the NA, NaN, or None values at the same position in the input series.
Conclusion
In this article, we have discussed different ways to check for nan values in pandas. To learn more about python programming,
you can read this article on how to sort a pandas dataframe. You might also like this article on how to drop columns from a pandas dataframe.
I hope you enjoyed reading this article. Stay tuned for more informative articles.
Happy Learning!
The post Check for NaN Values in Pandas Python appeared first on PythonForBeginners.com.
December 19, 2022 at 07:30PM
Click here for more details...
=============================
The original post is available in Planet Python by
this post has been published as it is through automation. Automation script brings all the top bloggers post under a single umbrella.
The purpose of this blog, Follow the top Salesforce bloggers and collect all blogs in a single place through automation.
============================
Post a Comment