5. DBFS (Databricks File System)

SIRIGIRI HARI KRISHNA
2 min readNov 24, 2023

--

DBFS

It is a distributed file system that allows you to write and read data files. Behind the scenes, the DBFS integrates with Azure Cloud storage such as Azure Blob and Azure Data Lake Gen2.It is a distributed file system just like HDFS.

1.DBFS makes it simple to work with cloud storage using folders and files.

DBFS is like a magic folder that makes it easy to store and find things in the cloud. No matter where you put your stuff or which cloud you use, DBFS organizes everything in a simple way. It’s like having a personal assistant that takes care of all the complicated storage details so you can just focus on finding and using your files easily.

2.Access data easily with DBFS, no need for credentials every time.

DBFS makes it easy to use data in the cloud without needing credentials each time. It takes care of credentials centrally, so you don’t have to bother with passwords or tokens in your code.

**As of now, DFS is still in use, and it has been replaced by Unity Catalog. Unity Catalog is more commonly used compared to DBFS.

In a real-time use case, imagine an organization that initially used DFS for file management across its network. Over time, they transitioned to Unity Catalog because it offered improved features, smoother integration with other tools, and a more user-friendly experience, making it the preferred choice for efficient file handling.

Please find the further article

Azure Databricks Access modes

https://sharikrishna26.medium.com/6-azure-databrick-accessmodes-686914c6e98f

--

--

SIRIGIRI HARI KRISHNA
SIRIGIRI HARI KRISHNA

Written by SIRIGIRI HARI KRISHNA

Data Engineer passionate about Spark, Azure, and the Cloud. Simplifying data complexities on my Medium blog. Let's dive into the world of data together!

No responses yet