Apache Spark SQL infrastructure boundary

Evidence-backed facts

1. Apache Spark SQL provides SQL, DataFrame, and Dataset abstractions for structured batch and streaming data across supported sources. informative
2. Lazy transformations define plans, while actions trigger distributed execution; writes, table changes, checkpoints, and cluster operations are separate effects. informative
3. Cluster manager, catalog, filesystem, table format, source credentials, code execution, configuration, and network policy bound actual access. informative
4. A logical plan or successful job does not prove source correctness, deterministic output, exactly-once effects, complete compatibility, or secure deployment. informative

Primary sources