Apache Arrow 0.15.0 was released on the 5th of October, 2019, which Koalas depends on to execute Pandas UDF, but the Spark community reports an issue with PyArrow 0.15.
We decided to set an upper bound for pyarrow version to avoid such issues until we are sure that Koalas works fine with it.
Set an upper bound for pyarrow version. (#918)
We continue improving multi-index columns support. We made the following APIs support multi-index columns:
pivot_table (#908)
pivot_table
melt (#920)
melt
We added the following new features:
koalas.DataFrame:
xs (#892)
xs
koalas.Series:
drop_duplicates (#896)
drop_duplicates
replace (#903)
replace
koalas.GroupBy:
shift (#910)
shift
Along with the following improvements:
Implement nested renaming for groupby agg (#904)
Add ‘index_col’ parameter to DataFrame.to_spark (#906)
Add more options to read_csv (#916)
read_csv
Add NamedAgg (#911)
Enable DataFrame setting value as list of labels (#905)