Data Geeks Hub
    Auto Loader
  • Auto Loader is a feature of Databricks
  • It basically detects new files or data and processes them automatically.
  • It can automatically handle changes in the schema of incoming data, such as new columns, without requiring manual intervention.
  • It can be used for both Batch and Streaming data.
Why?
Why was there a need to build an Auto Loader?
To automatically detect the new files or the modified files and process
Simplifying manual efforts.