[Python] Spartial Join in Apache Sedona

I want to test Apache Sedona with a spatial Join (like ST_Intersection, or gpd.Overlay).

1. Open GeoDataFrame A and B.
2. Count on both (to check both are working).(0.25 M, 0.5 M entries)
3. Indexing:

nutzung_RDD.analyze()
buildings_RDD.analyze()

nutzung_RDD.spatialPartitioning(GridType.KDBTREE)
buildings_RDD.spatialPartitioning(nutzung_RDD.getPartitioner())

1. Spatial Join (should be an intersection)

build_on_spatial_partitioned_rdd = True ## Set to TRUE only if run join query
using_index = True

try:
result_RDD = JoinQuery.SpatialJoinQueryFlat(buildings_RDD, nutzung_RDD, using_index, build_on_spatial_partitioned_rdd)
print(f"Number of results: {result_RDD.count()}")
print(result_RDD.take(5)) # Sample output
except Exception as e:
print(f"An error occurred during the spatial join: {e}")


I get the error:

An error occurred during the spatial join: An error occurred while calling zrg.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 15 in stage 52.0 failed 1 times, most recent failure: Lost task 15.0 in stage 52.0 (TID 541) (28dd096dafc7 executor driver): org.locationtech.jts.geom.TopologyException: side location conflict [ (463724.13, 5796494.38, NaN) ]


I am using Sedona:latest from dockerhub.
Normally, I would ask at gis.stackexchange, but it seams Sedona related question are more common here than there.

EDIT:

I rewrote the code, to use SQL instead:

result_gdf = sedona.sql("""SELECT ST_Intersection(b.geometry, u.geometry) AS geometry
FROM buildings b, usage u
WHERE ST_Intersects(b.geometry, u.geometry)"""
)
result_gdf.createOrReplaceTempView("result")

result2_gdf = sedona.sql("""
SELECT ST_MakeValid(geometry)
FROM result
""")
result2_gdf.count()


Result2 gets the similar error message:

24/10/07 19:08:00 ERROR Executor: Exception in task 6.0 in stage 173.0 (TID 2145) org.locationtech.jts.geom.TopologyException: side location conflict [ (407123.5427156649, 5803673.011053633, NaN) ]

I wonder why. Because I even tried ST_MakeValid.

EDIT 2: I even cannot show the problematic entries, because the error is also thrown for ST_Is_Valid.

result2_gdf = sedona.sql("""
SELECT building_oid
FROM result
WHERE ST_IsValid(geometry, 1) = false;
""")
result2_gdf.show()


org.apache.spark.sql.sedona_sql.expressions.InferredExpressionException:
Exception occurred while evaluating expression ST_Intersection - inputs:

[POLYGON ((354735.34 5758459.87, 354741.52 5758459.29,
354741.47 5758458.76, 354742.003 5758458.661, 354742.51 5758458.467, 354742.974 5758458.186, 354743.379 5758457.825, 354743.713 5758457.397, 354743.964 5758456.917, 354744.125 5758456.399, 354744.19 5758455.86, 354744.839 5758455.842, 354744.839 5758454.833, 354744.11 5758454.92, 354743.905 5758454.454, 354743.625 5758454.029, 354743.276 5758453.658, 354742.87 5758453.351, 354742.418 5758453.117, 354741.933 5758452.963, 354741.429 5758452.894, 354740.92 5758452.91, 354740.87 5758452.33, 354734.69 5758452.91, 354734.47 5758450.53, 354719.262 5758452.06, 354719.872 5758458.208, 354718.444 5758458.366, 354719.11 5758464.36, 354735.59 5758462.51, 354735.34 5758459.87)), MULTIPOLYGON (((354742.48361941514 5758458.483279419, 354742.5175583665
5758458.466906712, 354742.5512882294 5758458.450107478, 354742.5848036437 5758458.432884384, 354742.5599997879 5758458.445518499, 354742.53506431903 5758458.457890828, 354742.51 5758458.47, 354742.50122503354 5758458.474463283, 354742.49243145384 5758458.478889782, 354742.48361941514 5758458.483279419)), ((354744.19 5758455.86, 354744.18363257905 5758455.978846373, 354744.173464127 5758456.097428019, 354744.15950506565 5758456.215623404, 354744.1743896288 5758456.097518557, 354744.18455998343 5758455.978914724, 354744.19 5758455.86)), ((354742.76916934864 5758458.3281861525, 354742.856506923 5758458.272293335, 354742.9418533804 5758458.213404956, 354743.02510602196 5758458.151591878, 354743.006871307 5758458.164586073, 354742.988501966 5758458.177389245, 354742.97 5758458.19, 354742.90448312287 5758458.238112903, 354742.83751848264 5758458.284189465, 354742.76916934864 5758458.3281861525)), ((354742.4807933091 5758453.143215798, 354742.36904797406
5758453.096683949, 354742.255514648 5758453.054702802, 354742.1403785658 5758453.01734085, 354742.23469866544 5758453.048476773, 354742.32793967956 5758453.082708925, 354742.42 5758453.12, 354742.44034595153 5758453.127524007, 354742.4606111444 5758453.135262895, 354742.4807933091 5758453.143215798)), ((354743.56067013886 5758453.949935026, 354743.47363280016
5758453.851548408, 354743.382367441 5758453.757070585, 354743.28704942006 5758453.666683088, 354743.380944933 5758453.758478318, 354743.4721767185 5758453.852921386, 354743.56067013886 5758453.949935026)), ((354741.50910225126 5758452.898420274, 354741.6329594108 5758452.913596253, 354741.75649912434 5758452.931170253, 354741.87967496796 5758452.95113567, 354741.7569187535 5758452.928421285, 354741.633323018 5758452.9108393155, 354741.50910225126 5758452.898420274)), ((354742.87 5758453.35, 354743.00843700604 5758453.446922913, 354743.143314634 5758453.548740796, 354743.2744598884 5758453.655323056, 354743.1454582713 5758453.545990117, 354743.01049506286 5758453.444107629, 354742.87 5758453.35), (354741.47 5758458.76, 354741.69640811626 5758458.730853419, 354741.9199317863 5758458.6845141575, 354742.1392674122 5758458.621252467, 354742.09311297646 5758458.635119098, 354742.04668400256 5758458.648036752, 354742 5758458.66, 354741.91386881284 5758458.682274899, 354741.8271822026 5758458.702279502, 354741.74 5758458.72, 354741.6503692072 5758458.735750736, 354741.5603475359 5758458.74908728, 354741.47 5758458.76)), ((354743.91 5758454.45, 354743.82046237413 5758454.307324911, 354743.7278918259 5758454.166904364, 354743.63 5758454.03, 354743.676666975 5758454.1000003815, 354743.7233338356 5758454.170000792, 354743.77 5758454.24, 354743.8166666031 5758454.309999824, 354743.8633327484 5758454.37999928, 354743.91 5758454.45), (354743.5687541657 5758457.608473395, 354743.7185425014 5758457.395727332, 354743.849262841 5758457.170761499, 354743.959924779 5758456.935280356, 354744.04968988255 5758456.691068032, 354744.0221770005 5758456.768298324, 354743.9922706891 5758456.844633698, 354743.96 5758456.92, 354743.9258946364 5758457.001157474, 354743.8892155034 5758457.081184673, 354743.85 5758457.16, 354743.8060313074 5758457.241553628, 354743.75934706524 5758457.321583757, 354743.71 5758457.4, 354743.6652958208 5758457.4710824955, 354743.6181956441 5758457.540600536, 354743.5687541657 5758457.608473395)))], cause: found non-noded intersection between LINESTRING ( 354741.47
5758458.76, 354741.69640811626 5758458.730853419 ) and LINESTRING ( 354741.6503692072 5758458.735750736, 354741.5603475359 5758458.74908728 ) [ (354741.5973395641, 5758458.743606979, NaN) ]

Continue reading...