In the data warehouse, SQL developers use Object Explorer to navigate and manage database objects. They can also add existing data warehouses, SQL analytics endpoints from lake houses, and mirrored databases to query across databases. This feature aims to simplify sorting, searching, filtering, pinning, and collapsing database objects across databases in object explorer for easier management.