[jozefbakus]

Overview

Map type was added to standardization

Release Notes

• Added Map type support

Related

Closes #126

Test evidence:

Cannot add screenshots, but this version of standardization was tested on real pipeline/data
Resulting schema contains:
| | |-- metaData: map (nullable = true)
| | | |-- key: string
| | | |-- value: string (valueContainsNull = true)
And data with no errors in errCol

[github-actions]

JaCoCo code coverage report - scala 2.12.20, spark 3.5.7

Metric (instruction)	Coverage	Threshold	Status
Overall	83.25%	80.0%	✅
Changed Files	91.43%	80.0%	✅

Report	Coverage (O/Ch)	Threshold (O/Ch)	Status (O/Ch)
spark-data-standardization Jacoco Report - spark:3.5.7 scala:2.12.20	83.25% / 91.43%	80.0% / 80.0%	✅/✅

File Path	Coverage	Threshold	Status
PlainSchemaGenerator.scala	91.06%	60.0%	✅
SchemaValidator.scala	78.8%	60.0%	✅
TypeParser.scala	93.77%	60.0%	✅
TypedStructField.scala	88.6%	60.0%	✅

---

Run 39249119798 · Event: pull_request

[github-actions]

JaCoCo code coverage report - scala 2.13.17, spark 4.1.2

Metric (instruction)	Coverage	Threshold	Status
Overall	78.24%	75.0%	✅
Changed Files	86.04%	80.0%	✅

Report	Coverage (O/Ch)	Threshold (O/Ch)	Status (O/Ch)
spark-data-standardization Jacoco Report - spark:4.1.2 scala:2.13.17	78.24% / 86.04%	75.0% / 80.0%	✅/✅

File Path	Coverage	Threshold	Status
PlainSchemaGenerator.scala	89.91%	60.0%	✅
SchemaValidator.scala	80.19%	60.0%	✅
TypeParser.scala	85.54%	60.0%	✅
TypedStructField.scala	88.93%	60.0%	✅

---

Run 39249119798 · Event: pull_request

[yruslan]

Looks good to me. Checked the code and tests

[dk1844]

LGTM (read the code)