[DONT MERGE] Connected component suite#477
[DONT MERGE] Connected component suite#477bjornjorgensen wants to merge 8 commits intographframes:masterfrom
Conversation
|
@bjornjorgensen Thanks so much for this PR! When I get home I will review and test it :) |
rjurney
left a comment
rjurney
left a comment
There was a problem hiding this comment.
Looks good, mow let me test it...
| val orderedEdges = edges.filter(col(SRC) =!= col(DST)) | ||
| .select(minValue(col(SRC), col(DST)).as(SRC), maxValue(col(SRC), col(DST)).as(DST)) | ||
| .distinct() | ||
| .distinct() // Add explicit parentheses for Scala 2.13 |
There was a problem hiding this comment.
@bjornjorgensen what does this mean? distinct() has parenthesis before this edit...
There was a problem hiding this comment.
yes, we don't need it but there are 8 warnings when we build it https://github.com/graphframes/graphframes/actions/runs/12726219880/job/35474344050#step:5:2278 and I was trying to fix some of them.
| val components = g.connectedComponents.run() | ||
| val expected = Set(Set(0L, 1L, 2L), Set(3L, 4L, 5L), Set(6L), Set(7L)) | ||
| assertComponents(components, expected) | ||
| withSparkConf( |
There was a problem hiding this comment.
@bjornjorgensen why is disabling AQE needed when you disable it in the connected components code itself?
There was a problem hiding this comment.
yes you are right. I have made a new PR that works #478
|
I have made a new PR #478 instead of this one |
2 participants
No description provided.