pandas 浮点运算和比较导致不良输出

rkttyhzu 于 2023-01-07 发布在其他

关注(0)|答案(2)|浏览(135)

考虑Pandas Dataframe df1：

df1 = pd.DataFrame({"Name":["Kevin","Peter","James","Jose","Matthew","Pattrick","Alexander"],"Number":[1,2,3,4,5,6,7],"Total":[495.2,432.5,'-',395.5,485.8,415,418.7],"Average_old":[86.57,83.97,'-',96.59,84.67,83.10,83.84],"Grade_old":['A','A','A','A+','A','A','A'],"Total_old":[432.8,419.8,'-',482.9,423.3,415,418.7]})

我使用以下公式计算了Average和Grade

df1["Average"] = df1["Total"].apply(lambda x: x/5 + 0.1 if x != "-" else "-")
df1["Grade"] = df1["Average"].apply((lambda x:'A+' if x!='-' and x>90 else 'A'))

因此df1变为
DF1

Name  Number  Total  Average_old Grade_old  Total_old  Average Grade
0      Kevin       1  495.2        86.57         A      432.8    99.14    A+
1      Peter       2  432.5        83.97         A      419.8    86.60     A
2      James       3      -            -         A          -        -     A
3       Jose       4  395.5        96.59        A+      482.9    79.20     A
4    Matthew       5  485.8        84.67         A      423.3    97.26    A+
5   Pattrick       6  415.0        83.10         A      415.0    83.10     A
6  Alexander       7  418.7        83.84         A      418.7    83.84     A

df1具有Total, Total_old, Grade, Grade_old, Average, Average_old。我正在尝试检查Total的任何值是否相对于Total_old被修改，Grade的任何值相对于Grade_old被修改，或者Average的任何值相对于Average_old被修改。我尝试使用以下代码创建一个新的Dataframedfmod，它将给出df1的所有修改值

dfmod = pd.DataFrame()
columns =["Total","Average","Grade"]   
for col in columns:      
   dfmod = pd.concat([dfmod,df1[["Name","Number",col + '_old']][df1[col].ne(df1[col +'_old'])].dropna()],sort=False)

dfmod.rename(columns={col + '_old':col},inplace=True)           
dfmod = dfmod.groupby('Name',as_index = False,sort = False).first()

得到的输出为
dfmod

Name  Number  Total  Average Grade
0      Kevin       1  432.8    86.57     A
1      Peter       2  419.8    83.97  None
2       Jose       4  482.9    96.59    A+
3    Matthew       5  423.3    84.67     A
4  Alexander       7    NaN    83.84  None

此处，在比较Total与Total_old、Average与Average_old以及Grade与Grade_old时，未修改Pattrick的值，因此正确删除了Pattrick的条目。
但是如果你观察Alexander'sAverage即使Total，Average和Grade分别和Total_old,Average_old,Grade_old相同修改值 Dataframe dfmod具有作为修改值错误添加的Average值。发生这种情况的原因是因为浮点运算在编程语言中的工作方式与整数运算不同，如下面的链接所述。https://www.geeksforgeeks.org/floating-point-error-in-python/
所以我尝试将np.isclose函数实现为：

for col in columns:  
     if col is 'Grade':
          dfmod = pd.concat([dfmod,df1[["Name","Number",col + '_old']][df1[col].ne(df1[col +'_old'])].dropna()],sort=False)
       continue

    dfmod = pd.concat([dfmod,df1[["Name","Number",col + '_old']][~np.isclose(df1[col],df1[col+'_old'])].dropna()],sort=False)

但是它抛出错误消息

`Exception has occurred: TypeError ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''`

错误似乎是因为数据中的"-"字符，我如何才能修复这个问题，请做帮助，我在这个问题上卡住了一段时间，并尝试了所有的资源，我可以得到。
预期产出：

Name  Number  Total  Average Grade
0      Kevin       1  432.8    86.57     A
1      Peter       2  419.8    83.97     A
3       Jose       4  482.9    96.59    A+
4    Matthew       5  423.3    84.67     A

它应忽略James、Pattrick和Alexander的值，因为它们在Total-Total_old、Average-Average_old、Grade-Grade_old方面没有任何变化

pandas

来源：https://stackoverflow.com/questions/75013379/floating-point-arithmetic-and-comparison-causing-undesirable-output

2条答案

按热度按时间

vfh0ocws1#

据我所知，"-"字符是不必要的--你可以在它们出现的列中用None替换它，然后把这些列变成数字，这将使你的预处理步骤更加清晰，并避免不必要的条件语句，我们需要检查某些值是否是"-"。

df1 = df1.replace("-",None).astype({"Total":float, "Average_old":float, "Total_old":float})

这样，在创建新列Average和Grade时，就不必使用.apply或检查特定元素是否为"-"：

df1["Average"] = df1["Total"]/5 + 0.1
df1["Grade"] = ["A+" if x>90 else "A" for x in df1["Average"]]

然后，您可以在条件中使用np.isclose，删除任何包含null的行，选择包含"_old"的列，并重命名这些列：

condition = ~(np.isclose(df1['Average'], df1['Average_old']) & np.isclose(df1['Total'], df1['Total_old']) & (df1['Grade_old'] == df1['Grade']))
cols = ['Name','Number'] + [col for col in df1.columns if "_old" in col]
df1.loc[condition, cols].dropna().rename(columns={'Total_old':'Total','Average_old':'Average','Grade_old':'Grade'})

结果：

Name  Number  Total  Average Grade
0    Kevin       1  432.8    86.57     A
1    Peter       2  419.8    83.97     A
3     Jose       4  482.9    96.59    A+
4  Matthew       5  423.3    84.67     A

赞(0）回复(0）举报 2023-01-07

ux6nzvsh2#

如果这是你要找的，请看一下。

import pandas as pd
import numpy as np

def compute_grade(new_average, old_grade):
    try:
        grade = 'A+' if float(new_average) > 90 else 'A'
    except:
        grade = old_grade
    return grade

df1 = pd.DataFrame({"Name":["Kevin","Peter","James","Jose","Matthew","Pattrick","Alexander"],"Number":[1,2,3,4,5,6,7],"Total":[495.2,432.5,'-',395.5,485.8,415,418.7],"Average_old":[86.57,83.97,'-',96.59,84.67,83.10,83.84],"Grade_old":['A','A','A','A+','A','A','A'],"Total_old":[432.8,419.8,'-',482.9,423.3,415,418.7]})
df1["Average"] = df1["Total"].apply(lambda x: round((x/5) + 0.1, 2) if x != "-" else "-")
df1["Grade"] = df1.apply((lambda x: compute_grade(x['Average'], x['Grade_old'])), axis=1)

print(df1)

# import pdb; pdb.set_trace()

dfmod = df1[(df1['Total'] != df1['Total_old']) | (df1['Average'] != df1['Average_old']) | (df1['Grade'] != df1['Grade_old'])]

print(dfmod)

输出：

Name  Number  Total Average_old Grade_old Total_old Average Grade
0      Kevin       1  495.2       86.57         A     432.8   99.14    A+
1      Peter       2  432.5       83.97         A     419.8    86.6     A
2      James       3      -           -         A         -       -     A
3       Jose       4  395.5       96.59        A+     482.9    79.2     A
4    Matthew       5  485.8       84.67         A     423.3   97.26    A+
5   Pattrick       6    415        83.1         A       415    83.1     A
6  Alexander       7  418.7       83.84         A     418.7   83.84     A
      Name  Number  Total Average_old Grade_old Total_old Average Grade
0    Kevin       1  495.2       86.57         A     432.8   99.14    A+
1    Peter       2  432.5       83.97         A     419.8    86.6     A
3     Jose       4  395.5       96.59        A+     482.9    79.2     A
4  Matthew       5  485.8       84.67         A     423.3   97.26    A+

将我们计算的平均值四舍五入到2位小数是这里的关键。
此外，在计算Grade时，如果遇到非数字值（如"-"），则返回Grade_old。

赞(0）回复(0）举报 2023-01-07

我来回答

pandas 浮点运算和比较导致不良输出

2条答案

相关问题

热门标签

最新问答