Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Oct 28, 2022

• [SPARK-40512][SPARK-40896][PS][INFRA] Upgrade pandas to 1.5.0

  Show description for cf086b1
  

  itholic

  authored and

  zhengruifeng

  committed
  cf086b1

  Copy full SHA for cf086b1

Commits on Oct 27, 2022

• [SPARK-40922][PYTHON] Document multiple path support in `pyspark.pandas.read_csv`

  Show description for 5b5eb23
  

  soxofaan

  authored and

  HyukjinKwon

  committed
  5b5eb23

  Copy full SHA for 5b5eb23

Commits on Oct 25, 2022

• [SPARK-40879][CONNECT] Support Join UsingColumns in proto

  Show description for a27ccd7
  

  Rui Wang

  authored and

  cloud-fan

  committed
  a27ccd7

  Copy full SHA for a27ccd7

• [SPARK-40907][PS][SQL] PandasMode` should copy keys before inserting into Map

  Show description for e4d0412
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  e4d0412

  Copy full SHA for e4d0412

• [SPARK-40836][CONNECT] AnalyzeResult should use struct for schema

  Show description for 0b3d954
  

  Rui Wang

  authored and

  cloud-fan

  committed
  0b3d954

  Copy full SHA for 0b3d954

• [SPARK-40899][CONNECT] Make UserContext extensible

  Show description for e64afb6
  

  grundprinzip

  authored and

  cloud-fan

  committed
  e64afb6

  Copy full SHA for e64afb6

Commits on Oct 24, 2022

• [SPARK-40897][DOCS] Add some PySpark APIs to References

  Show description for e2e449e
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  e2e449e

  Copy full SHA for e2e449e

• [SPARK-34265][PYTHON][SQL] Instrument Python UDFs using SQL metrics

  Show description for e966c38
  

  LucaCanali

  authored and

  cloud-fan

  committed
  e966c38

  Copy full SHA for e966c38

• [SPARK-40812][CONNECT][PYTHON][FOLLOW-UP] Improve Deduplicate in Python client

  Show description for 74c8264
  

  Rui Wang

  authored and

  cloud-fan

  committed
  74c8264

  Copy full SHA for 74c8264

Commits on Oct 23, 2022

• [SPARK-40821][SQL][CORE][PYTHON][SS] Introduce window_time function to extract event time from the window column

  Show description for 96b5d50
  

  alex-balikov

  authored and

  HeartSaVioR

  committed
  96b5d50

  Copy full SHA for 96b5d50

Commits on Oct 22, 2022

• [SPARK-40874][PYTHON] Fix broadcasts in Python UDFs when encryption enabled

  Show description for 8a96f69
  

  peter-toth

  authored and

  HyukjinKwon

  committed
  8a96f69

  Copy full SHA for 8a96f69

• [SPARK-40796][CONNECT][DOC][FOLLOW-UP] Add check command in Readme

  Show description for 6545a08
  zhengruifeng

  committed
  6545a08

  Copy full SHA for 6545a08

• [SPARK-40796][CONNECT][FOLLOW-UP] Improve README for proto generated files in Connect Python client

  Show description for 98f9eda
  

  Rui Wang

  authored and

  zhengruifeng

  committed
  98f9eda

  Copy full SHA for 98f9eda

Commits on Oct 21, 2022

• [SPARK-40854][CONNECT] Use proper JSON encoding until we have Arrow collection

  Show description for 26e258c
  

  grundprinzip

  authored and

  HyukjinKwon

  committed
  26e258c

  Copy full SHA for 26e258c

• [SPARK-40839][CONNECT][PYTHON] Implement `DataFrame.sample`

  Show description for 7934f00
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  7934f00

  Copy full SHA for 7934f00

• [SPARK-40657] Add support for Java classes in Protobuf functions

  Show description for fd9e576
  

  Raghu Angadi

  authored and

  HeartSaVioR

  committed
  fd9e576

  Copy full SHA for fd9e576

• [SPARK-40812][CONNECT] Add Deduplicate to Connect proto and DSL

  Show description for b14da8b
  

  Rui Wang

  authored and

  cloud-fan

  committed
  b14da8b

  Copy full SHA for b14da8b

• [SPARK-40813][CONNECT][PYTHON][FOLLOW-UP] Improve limit and offset in Python client

  Show description for 45bb957
  

  Rui Wang

  authored and

  HyukjinKwon

  committed
  45bb957

  Copy full SHA for 45bb957

Commits on Oct 20, 2022

• [SPARK-40838][INFRA][TESTS] Upgrade infra base image to focal-20220922 and fix ps.mlflow doctest

  Show description for 2698d6b
  Yikun

  committed
  2698d6b

  Copy full SHA for 2698d6b

• [SPARK-40539][CONNECT] Initial DataFrame Read API parity for Spark Connect

  Show description for 01c7a46
  

  Rui Wang

  authored and

  cloud-fan

  committed
  01c7a46

  Copy full SHA for 01c7a46

Commits on Oct 19, 2022

• [SPARK-40587][CONNECT][FOLLOW-UP] Make sure python client support select *

  Show description for 3c5bc21
  

  Rui Wang

  authored and

  cloud-fan

  committed
  3c5bc21

  Copy full SHA for 3c5bc21

• [SPARK-40813][CONNECT] Add limit and offset to Connect DSL

  Show description for d7265d6
  

  Rui Wang

  authored and

  HyukjinKwon

  committed
  d7265d6

  Copy full SHA for d7265d6

• [SPARK-40779][PS][TEST] Fix `corrwith` to work properly with different anchor

  Show description for 5b61227
  

  itholic

  authored and

  HyukjinKwon

  committed
  5b61227

  Copy full SHA for 5b61227

• [SPARK-40823][CONNECT][FOLLOW-UP] Fix test_connect_column_expressions.py

  Show description for 74be7e6
  

  zhengruifeng

  authored and

  HyukjinKwon

  committed
  74be7e6

  Copy full SHA for 74be7e6

• [SPARK-40538][CONNECT][FOLLOW-UP] Fix less than comparison in Spark Connect expression

  Show description for 7ea33b7
  HyukjinKwon

  committed
  7ea33b7

  Copy full SHA for 7ea33b7

• [SPARK-40655][PYTHON][PROTOBUF] PySpark support for from_protobuf and to_protobuf

  Show description for ed9db14
  

  SandishKumarHN

  authored and

  HyukjinKwon

  committed
  ed9db14

  Copy full SHA for ed9db14

• [SPARK-40823][CONNECT] Connect Proto should carry unparsed identifiers

  Show description for 14d8604
  

  Rui Wang

  authored and

  cloud-fan

  committed
  14d8604

  Copy full SHA for 14d8604

• [SPARK-40816][CONNECT][PYTHON] Rename LogicalPlan.collect to LogicalPlan.to_proto

  Show description for 646d716
  

  Rui Wang

  authored and

  HyukjinKwon

  committed
  646d716

  Copy full SHA for 646d716

• [SPARK-40828][CONNECT][PYTHON][TESTING] Drop Python test tables before and after unit tests

  Show description for 7af39b6
  

  Rui Wang

  authored and

  HyukjinKwon

  committed
  7af39b6

  Copy full SHA for 7af39b6

Commits on Oct 18, 2022

• [SPARK-40538][CONNECT] Improve built-in function support for Python client

  Show description for a9da924
  

  grundprinzip

  authored and

  HyukjinKwon

  committed
  a9da924

  Copy full SHA for a9da924

• [SPARK-40774][CONNECT] Add Sample to proto and Connect DSL

  Show description for d0ab83c
  

  Rui Wang

  authored and

  cloud-fan

  committed
  d0ab83c

  Copy full SHA for d0ab83c

• [SPARK-40809][SPARK-40780][FOLLOW-UP] Improve filter and alias testing coverage in python client

  Show description for 15dc0eb
  

  Rui Wang

  authored and

  HyukjinKwon

  committed
  15dc0eb

  Copy full SHA for 15dc0eb

• [SPARK-40796][BUILD][FOLLOW-UP] Fix Mypy check on unused "type: ignore"

  Show description for e4e8148
  

  Rui Wang

  authored and

  HyukjinKwon

  committed
  e4e8148

  Copy full SHA for e4e8148

• [SPARK-40743][CONNECT] StructType should contain a list of StructField and each field should have a name

  Show description for 269acf7
  

  Rui Wang

  authored and

  cloud-fan

  committed
  269acf7

  Copy full SHA for 269acf7

Commits on Oct 17, 2022

• [SPARK-40556][PS][SQL] AQE clean up resources after each stage and eagerly clean Pandas Index Cache

  Show description for 4279090
  zhengruifeng

  committed
  4279090

  Copy full SHA for 4279090